TLDR; We want to log every order line item from stores using our app and properly attribute which line items were ordered through our extensions – currently we’re not logging all the line items correctly and debug console.logs are not being returned in vercel logs. Can you please help advise me?
Webhook Logging Overview for App
Objective:
To add detailed diagnostic logging to the Shopify orders/create
webhook handler (/api/webhooks
endpoint, handled by pages/api/webhooks/orders/create.ts
) to debug issues related to:
- Line Item Attribution: Incorrectly assigning or failing to assign
recommendation_source
andrecommendation_type
to line items added via Platter+ extensions (Checkout UI extensions, Post-Purchase). - Product Handle Extraction: Failing to correctly extract and save the product
handle
associated with Platter+ interactions, particularly when using the_platter_sales
order note attribute. - Data Saving Errors: Investigating potential database errors (like Prisma
P2000
- value too long for column type) during the saving of order line items. - Missing Line Items: Understanding why some line items present in the Shopify order payload might not be getting saved to the database.
Code Location:
The primary file containing this enhanced logging logic is: pages/api/webhooks/orders/create.ts
Logging Strategy:
We’ve implemented detailed console.log
statements at various key points within the webhook handler function. These logs are prefixed with DEBUG:
or [HANDLE DEBUG]:
for easier filtering and identification in log streams (e.g., Vercel logs).
Key Logging Points & Code Snippets:
-
Handler Entry Confirmation: To verify the correct version of the code is being executed.
// --- SUPER DEBUG LOG --- console.log("--- ORDERS/CREATE.TS HANDLER ENTRY POINT V3 ---"); // --- END SUPER DEBUG LOG --- // ... near the start of the handler function ... console.log("[HANDLE DEBUG] orders/create handler entered - v2");
-
Raw Line Item Properties: To see the exact properties received from Shopify for each line item before any processing.
// Inside the .map() loop iterating over order.line_items console.log(`[${eventId}] RAW PROPERTIES for line item ${item.id}:`, JSON.stringify(properties));
-
Basic Line Item Processing Info: To confirm extraction of recommendation type/source using utility functions.
// After extracting properties, type, source console.log( `DEBUG: Processing Line item ${item.id} (${item.title}) - type: ${ recommendationType || "N/A" }, source: ${recommendationSource || "N/A"}, props:`, platterProps, // Logs properties specifically extracted by extractPlatterProperties );
-
_platter_sales
Note Attribute Parsing & Handle Lookup (Method 1): To debug handle extraction from the primary note attribute method.// Inside the .map() loop, within the try block for Method 1 const rawPlatterSalesAttr = order.note_attributes?.find(/* ... */); // Log raw value of the note attribute console.log(`DEBUG: _platter_sales for Order ${order.id}, Line Item ${item.id}: ${rawPlatterSalesAttr?.value || "NOT FOUND"}`); console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Raw Note Attr (_platter_sales): ${rawPlatterSalesAttr ? JSON.stringify(rawPlatterSalesAttr) : 'Not Found'}`); if (rawPlatterSalesAttr?.value) { const salesDataMap = JSON.parse(rawPlatterSalesAttr.value) /* ... */; // Log the fully parsed JSON map console.log(`DEBUG: Parsed _platter_sales map for Order ${order.id}: ${JSON.stringify(salesDataMap, null, 2)}`); console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Parsed Sales Map: ${JSON.stringify(salesDataMap)}`); const platterSaleKey = formattedProperties["_platter_sale"]; // Log the key used for lookup (extracted from line item properties) console.log(`DEBUG: Looking for platterSaleKey: "${platterSaleKey}" for Line Item ${item.id}`); console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Platter Sale Key: '${platterSaleKey}'`); if (platterSaleKey && salesDataMap && salesDataMap[platterSaleKey]) { // Log the entry found in the map console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Found Map Entry: ${JSON.stringify(salesDataMap[platterSaleKey])}`); // Log if a handle was successfully extracted console.log(`DEBUG: Found handle "${userDefinedHandle}" for key "${platterSaleKey}" in Line Item ${item.id}`); } else { // Log if the lookup failed console.log(`DEBUG: Handle lookup FAILED for key "${platterSaleKey}" in Line Item ${item.id}`); console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Key '${platterSaleKey}' not found in Sales Map or Map empty.`); } } // Catch and log any JSON parsing errors console.error(`[Handle Parsing Error] Order: ${order.id}, Line: ${item.id}, Error: ${e.message}`);
-
Direct Handle Property Fallback (Method 2): To debug handle extraction using the
__platter_extension_handle
line item property.// Inside the .map() loop, if Method 1 didn't find a handle if (!userDefinedHandle) { const directHandle = formattedProperties["__platter_extension_handle"]; // Log the value found (or not found) in the direct property console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Direct Handle Property Value: '${directHandle}'`); if (directHandle /* ... */) { // Log if the fallback succeeded console.log(`[Handle Fallback Succeeded] Order: ${order.id}, Line: ${item.id}, Using Direct Handle: '${userDefinedHandle}'`); } else { // Log if the fallback value wasn't usable console.log(`[Handle Fallback Failed] Order: ${order.id}, Line: ${item.id}, Direct Handle not usable.`); } }
-
Final Handle Determination: To see the final handle value assigned to the line item data.
// Inside the .map() loop, after both methods have been attempted if (userDefinedHandle) { console.log(`[Handle Found] Order: ${order.id}, Line: ${item.id}, Handle: '${userDefinedHandle}'`); } else { console.log(`[Handle Not Found] Order: ${order.id}, Line: ${item.id}`); }
-
Complete Line Item Data Object: To log the exact data object being prepared for each line item before it’s added to the array for database insertion. This helps identify potential issues (like incorrect values or overly long strings) before the save attempt.
// Inside the .map() loop, just before returning the final object const finalLineItemData = { /* ... all fields ... */ }; console.log(`DEBUG: Complete line item data for ${item.id}: ${JSON.stringify(finalLineItemData, null, 2)}`); return finalLineItemData;
-
Data for
createMany
: To log the final array of all processed line item objects being sent to theprisma.order_line_items.createMany
database call.// Before the prisma.order_line_items.createMany call console.log("DEBUG: Data for prisma.order_line_items.createMany:", JSON.stringify(lineItemsData, null, 2));
Current Status:
Despite multiple attempts to deploy the code containing these logs, they are consistently not appearing in our Vercel environment logs for the relevant webhook invocations. This is preventing further diagnosis of the underlying issues (incomplete _platter_sales
data generation, missing line items in DB, P2000 errors). We are currently investigating potential deployment/build issues or webhook configuration problems as the next step.
Full web hook create code:
import { NextApiRequest, NextApiResponse } from “next”;
import { PrismaClient, Prisma } from “@prisma/client”;
import { buffer } from “micro”;
import { createHmac, timingSafeEqual } from “crypto”;
import {
detectRecommendationType,
extractRecommendationSource,
formatLineItemProperties,
extractPlatterProperties,
} from “../../../../utils/recommendation-utils”;
import {
isEventProcessed,
markEventProcessed,
logWebhookMetrics,
ShopifyOrder,
ShopifyLineItem,
} from “../../../../utils/webhook-utils”;
// Define the intermediate type for processed line items
interface ProcessedLineItem {
shop_domain: string;
shopify_line_item_id: string;
shopify_order_id: string;
product_id: string | null;
variant_id: string | null;
title: string;
quantity: number;
price: Prisma.Decimal;
sku: string; // Allow empty string based on code
vendor: string | null;
recommendation_type: string | null;
recommendation_source: string | null;
created_at: Date;
updated_at: Date;
handle?: string | null;
}
// Initialize Prisma client with explicit database URL
const prisma = new PrismaClient({
datasources: {
db: {
url:
process.env.NODE_ENV === “production”
? process.env.DATABASE_URL
: process.env.DATABASE_URL_DEV,
},
},
});
const SHOPIFY_API_SECRET = process.env.SHOPIFY_WEBHOOK_KEY || “”;
// Helper function (assuming it’s defined elsewhere or kept)
function safeAccess(obj: unknown, path: string, defaultValue: T): T {
// Implementation omitted for brevity…
if (!obj || typeof obj !== “object”) return defaultValue;
const keys = path.split(“.”);
let current: any = obj;
for (const key of keys) {
if (current === null || current === undefined || typeof current !== “object”) {
return defaultValue;
}
current = current[key];
}
return current !== null && current !== undefined ? (current as T) : defaultValue;
}
// Disable the default body parser to get the raw body
export const config = {
api: {
bodyParser: false,
},
};
/**
- Verify Shopify webhook signature
*/
function verifyWebhookSignature(
body: string,
hmacHeader: string | string | undefined
): boolean {
if (!hmacHeader || Array.isArray(hmacHeader)) {
return false;
}
const generatedHash = createHmac(“sha256”, SHOPIFY_API_SECRET)
.update(body, “utf8”)
.digest(“base64”);
return timingSafeEqual(Buffer.from(generatedHash), Buffer.from(hmacHeader));
}
/**
- Webhook handler for order creation events from Shopify
*/
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
// Add a very early log to confirm execution of this version
console.log(“[HANDLE DEBUG] orders/create handler entered - v2”);
// — SUPER DEBUG LOG —
console.log(“— ORDERS/CREATE.TS HANDLER ENTRY POINT V3 —”);
// — END SUPER DEBUG LOG —
const startTime = Date.now();
const topic = “orders/create”;
let shopDomain = “”;
let eventId = “”;
let success = false; // Initialize success as false
// Only allow POST requests
if (req.method !== “POST”) {
return res.status(405).json({ error: “Method not allowed” });
}
try {
// Get the raw body
const rawBody = await buffer(req);
const body = rawBody.toString();
// Get the headers
const hmacHeader = req.headers["x-shopify-hmac-sha256"] as string;
shopDomain = req.headers["x-shopify-shop-domain"] as string;
eventId =
(req.headers["x-shopify-webhook-id"] as string) ||
(req.headers["x-shopify-event-id"] as string) ||
`${shopDomain}-${Date.now()}`;
console.log(`Received webhook at /api/webhooks/orders/create`, {
shopDomain,
eventId,
hmacHeader: hmacHeader ? `${hmacHeader.substring(0, 10)}...` : "none",
bodyLength: body.length,
});
// Verify the webhook signature
if (!verifyWebhookSignature(body, hmacHeader)) {
console.error("Invalid webhook signature");
logWebhookMetrics(topic, shopDomain, eventId, false, startTime);
return res.status(401).json({ error: "Invalid signature" });
}
// Check if we've already processed this event
if (isEventProcessed(eventId)) {
console.log(`Event ${eventId} already processed, acknowledging receipt`);
logWebhookMetrics(topic, shopDomain, eventId, true, startTime);
return res.status(200).json({ success: true, status: "already_processed" });
}
// Mark this event as processed EARLY - before database operation
markEventProcessed(eventId);
// Parse the order data
const order = JSON.parse(body) as ShopifyOrder;
// --- Start Inlined Synchronous Logic ---
try {
console.log(
`Synchronously processing order ${order.id} for shop ${shopDomain} (Event ID: ${eventId})`
);
const now = new Date();
// Map line items
const lineItemsData = (order.line_items || []).map((item: any) => {
// Format price as Decimal
const priceString = item.price?.toString() || "0";
// Extract Platter-related properties from line_item
const formattedProperties = formatLineItemProperties(item.properties || []);
const recommendationType = detectRecommendationType(formattedProperties);
const recommendationSource = extractRecommendationSource(formattedProperties);
const platterProps = extractPlatterProperties(item);
// --- Start Enhanced Logging ---
console.log(
`DEBUG: Processing Line item ${item.id} (${item.title}) - type: ${
recommendationType || "N/A"
}, source: ${recommendationSource || "N/A"}, props:`,
platterProps,
);
// --- End Enhanced Logging ---
let userDefinedHandle: string | null = null;
// --- METHOD 1: Note Attribute Parsing (Primary method) ---
try {
const rawPlatterSalesAttr = order.note_attributes?.find((p: any) => p.name === "_platter_sales");
// Log the raw attribute found (or lack thereof)
// --- Start Enhanced Logging ---
console.log(`DEBUG: _platter_sales for Order ${order.id}, Line Item ${item.id}: ${rawPlatterSalesAttr?.value || "NOT FOUND"}`);
// --- End Enhanced Logging ---
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Raw Note Attr (_platter_sales): ${rawPlatterSalesAttr ? JSON.stringify(rawPlatterSalesAttr) : 'Not Found'}`);
if (rawPlatterSalesAttr?.value) {
const salesDataMap = JSON.parse(rawPlatterSalesAttr.value) as Record<
string,
{ handle?: string; source?: string; extension?: string }
>;
// Log the parsed map
// --- Start Enhanced Logging ---
console.log(`DEBUG: Parsed _platter_sales map for Order ${order.id}: ${JSON.stringify(salesDataMap, null, 2)}`);
// --- End Enhanced Logging ---
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Parsed Sales Map: ${JSON.stringify(salesDataMap)}`);
const platterSaleKey = formattedProperties["_platter_sale"];
// Log the key being used for lookup
// --- Start Enhanced Logging ---
console.log(`DEBUG: Looking for platterSaleKey: "${platterSaleKey}" for Line Item ${item.id}`);
// --- End Enhanced Logging ---
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Platter Sale Key: '${platterSaleKey}'`);
if (platterSaleKey && salesDataMap && salesDataMap[platterSaleKey]) {
// Log the object found in the map
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Found Map Entry: ${JSON.stringify(salesDataMap[platterSaleKey])}`);
const foundHandle = salesDataMap[platterSaleKey].handle;
if (foundHandle && typeof foundHandle === 'string' && foundHandle.trim() !== '') {
userDefinedHandle = foundHandle.trim();
// --- Start Enhanced Logging ---
console.log(`DEBUG: Found handle "${userDefinedHandle}" for key "${platterSaleKey}" in Line Item ${item.id}`);
// --- End Enhanced Logging ---
}
} else {
// --- Start Enhanced Logging ---
console.log(`DEBUG: Handle lookup FAILED for key "${platterSaleKey}" in Line Item ${item.id}`);
// --- End Enhanced Logging ---
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Key '${platterSaleKey}' not found in Sales Map or Map empty.`);
}
}
} catch (e: any) {
// --- Start Enhanced Logging ---
console.error(`[Handle Parsing Error] Order: ${order.id}, Line: ${item.id}, Error: ${e.message}`);
// --- End Enhanced Logging ---
}
// --- METHOD 2: Direct Property Fallback ---
if (!userDefinedHandle) { // Only check if Method 1 failed
const directHandle = formattedProperties["__platter_extension_handle"];
// Log the direct handle value found (or lack thereof)
console.log(`[Handle Debug] Order: ${order.id}, Line: ${item.id}, Direct Handle Property Value: '${directHandle}'`);
if (directHandle && typeof directHandle === 'string' && directHandle.trim() !== '') {
userDefinedHandle = directHandle.trim();
console.log(`[Handle Fallback Succeeded] Order: ${order.id}, Line: ${item.id}, Using Direct Handle: '${userDefinedHandle}'`);
} else {
console.log(`[Handle Fallback Failed] Order: ${order.id}, Line: ${item.id}, Direct Handle not usable.`);
}
}
// --- Log Final Handle Determination ---
if (userDefinedHandle) {
console.log(`[Handle Found] Order: ${order.id}, Line: ${item.id}, Handle: '${userDefinedHandle}'`);
} else {
console.log(`[Handle Not Found] Order: ${order.id}, Line: ${item.id}`);
}
// Prepare the final data object for this line item
const finalLineItemData = {
shop_domain: shopDomain,
shopify_order_id: (order.id || "").toString(),
shopify_line_item_id: (item.id || "").toString(),
product_id: item.product_id ? item.product_id.toString() : null,
variant_id: item.variant_id ? item.variant_id.toString() : null,
title: item.title || item.name || "",
handle: userDefinedHandle, // Save the result of the tiered logic
quantity: item.quantity || 0,
price: new Prisma.Decimal(priceString),
sku: item.sku || "",
vendor: shopDomain, // Assign shopDomain instead of item.vendor
recommendation_type: recommendationType,
recommendation_source: recommendationSource ? recommendationSource.substring(0, 50) : null,
created_at: new Date(),
updated_at: new Date(),
};
// --- Start Enhanced Logging ---
// Log the complete line item data before returning
console.log(`DEBUG: Complete line item data for ${item.id}: ${JSON.stringify(finalLineItemData, null, 2)}`);
// --- End Enhanced Logging ---
return finalLineItemData;
});
const hasCrosssells = lineItemsData.some(
(item) => item.recommendation_type === "cross-sell"
);
const hasUpsells = lineItemsData.some(
(item) => item.recommendation_type === "upsell"
);
const hasPlatterInteraction = hasCrosssells || hasUpsells;
// Collect unique handles and surfaces from line items
const uniqueHandles = new Set<string>();
const uniqueSources = new Set<string>(); // To keep original source for platter_extensions
const uniqueSurfaces = new Set<string>();
lineItemsData.forEach(item => {
if (item.handle) { // User-defined handle
uniqueHandles.add(item.handle);
}
if (item.recommendation_source) { // Original source/type
uniqueSources.add(item.recommendation_source);
}
// Determine surface
let itemSurface = "CHECKOUT"; // Default
if (item.recommendation_source?.includes("post-purchase")) {
itemSurface = "POST_PURCHASE";
} else if (item.recommendation_source?.includes("cart")) {
itemSurface = "IN_CART";
}
uniqueSurfaces.add(itemSurface);
});
const platterHandlesArray = Array.from(uniqueHandles);
const platterSourcesArray = Array.from(uniqueSources); // Use this for platter_extensions
const surfacesArray = Array.from(uniqueSurfaces);
// Keep this logging for debugging
console.log(
`DEBUG: Order ${order.id} interactions - crossSells: ${hasCrosssells}, upsells: ${hasUpsells}, platter: ${hasPlatterInteraction}, handles: [${platterHandlesArray.join(', ')}], sources: [${platterSourcesArray.join(', ')}], surfaces: [${surfacesArray.join(', ')}]`
);
// Step 1: Use upsert instead of create
console.log(`Step 1: Upserting order record ${order.id}`);
const upsertedOrder = await prisma.orders.upsert({
where: {
// Specify the unique constraint fields
shop_domain_shopify_order_id: {
shop_domain: shopDomain,
shopify_order_id: (order.id || "").toString(),
},
},
update: {
// Fields to update if the order already exists
order_name: order.name || "",
email: order.email || "",
total_price: new Prisma.Decimal(order.total_price?.toString() || "0"),
subtotal_price: new Prisma.Decimal(order.subtotal_price?.toString() || "0"),
total_tax: new Prisma.Decimal(order.total_tax?.toString() || "0"),
currency: order.currency || "USD",
financial_status: order.financial_status || null,
fulfillment_status: order.fulfillment_status || null,
updated_at: new Date(order.updated_at || now.toISOString()), // Ensure updated_at is always set
processed_at: order.processed_at ? new Date(order.processed_at) : null,
cancelled_at: order.cancelled_at ? new Date(order.cancelled_at) : null,
cancel_reason: order.cancel_reason || null,
tags: order.tags || [],
note: order.note || "",
has_crosssells: hasCrosssells,
has_upsells: hasUpsells,
has_platter_interaction: hasPlatterInteraction,
platter_extensions: platterSourcesArray.length > 0 ? platterSourcesArray : [], // Keep original sources here
platter_handles: platterHandlesArray.length > 0 ? platterHandlesArray : [], // Add handles array
surfaces: surfacesArray.length > 0 ? surfacesArray : [], // Add surfaces array
},
create: {
// Fields to set when creating a new order
shop_domain: shopDomain,
shopify_order_id: (order.id || "").toString(),
order_name: order.name || "",
email: order.email || "",
total_price: new Prisma.Decimal(order.total_price?.toString() || "0"),
subtotal_price: new Prisma.Decimal(order.subtotal_price?.toString() || "0"),
total_tax: new Prisma.Decimal(order.total_tax?.toString() || "0"),
currency: order.currency || "USD",
financial_status: order.financial_status || null,
fulfillment_status: order.fulfillment_status || null,
created_at: new Date(order.created_at || now.toISOString()), // Set created_at only on create
updated_at: new Date(order.updated_at || now.toISOString()),
processed_at: order.processed_at ? new Date(order.processed_at) : null,
cancelled_at: order.cancelled_at ? new Date(order.cancelled_at) : null,
cancel_reason: order.cancel_reason || null,
tags: order.tags || [],
note: order.note || "",
has_crosssells: hasCrosssells,
has_upsells: hasUpsells,
has_platter_interaction: hasPlatterInteraction,
platter_extensions: platterSourcesArray.length > 0 ? platterSourcesArray : [], // Keep original sources here
platter_handles: platterHandlesArray.length > 0 ? platterHandlesArray : [], // Add handles array
surfaces: surfacesArray.length > 0 ? surfacesArray : [], // Add surfaces array
},
});
console.log(`Step 1: Order record ${upsertedOrder.id} upserted successfully.`);
// Step 2: Clear existing line items before adding new ones (important for idempotency)
console.log(`Step 2: Deleting existing line items for order ${order.id}`);
await prisma.order_line_items.deleteMany({
where: {
shopify_order_id: (order.id || "").toString(),
},
});
console.log(`Step 2: Existing line items deleted.`);
// Step 3: Create Line Items using createMany for better performance
if (lineItemsData.length > 0) {
console.log(`Step 3: Creating ${lineItemsData.length} new line items for order ${order.id}`);
// --- Start Enhanced Logging ---
// Log the data being sent to createMany
console.log("DEBUG: Data for prisma.order_line_items.createMany:", JSON.stringify(lineItemsData, null, 2));
// --- End Enhanced Logging ---
const createdLineItems = await prisma.order_line_items.createMany({
data: lineItemsData,
skipDuplicates: true, // Skip duplicates to avoid errors on retry
});
console.log(`Step 3: ${createdLineItems.count} new line items created successfully.`);
} else {
console.log(`Step 3: No new line items to create for order ${order.id}.`);
}
console.log(
`Order ${order.id} and its line items successfully processed and saved.`
);
success = true;
} catch (dbError) {
console.error(
`Synchronous DB processing error for order ${order?.id || "unknown"} (Event ID: ${eventId}):`,
dbError,
);
}
// --- End Inlined Logic ---
// Log metrics (reflects DB success/failure)
logWebhookMetrics(topic, shopDomain, eventId, success, startTime);
// Return success response (always 200 to Shopify)
return res.status(200).json({ success: true }); // Acknowledge receipt even if DB failed
} catch (error: unknown) {
// Outer catch block
const errorMessage = error instanceof Error
? error.message
: (error ? String(error) : “Unknown error occurred”);
const errorDetails = error instanceof Error
? { stack: error.stack }
: { value: error }; // Log the raw value if not an Error
// Log the error details more safely
console.error(
`[${eventId}] Error handling order created webhook (outer try): ${errorMessage}`,
errorDetails
);
// Log metrics (now definitely false)
try {
logWebhookMetrics(topic, shopDomain, eventId, false, startTime);
} catch (metricError) {
console.error(`[${eventId}] Failed to log metrics after outer error:`, metricError);
}
// Return 500 for critical errors
return res.status(500).json({
error: "Internal server error",
message: errorMessage, // Use the safe message
});
}
}
// — SUPER DEBUG LOG —
console.log(“— ORDERS/CREATE.TS HANDLER ENTRY POINT V3 —”);
// — END SUPER DEBUG LOG —