Hi Team - I followed the best practices via the webhooks guide but still seeing a 77% failure rate. Can someone please help me with the configuration?
Webhook Architecture: Comprehensive Overview
Our webhook system implements a robust, production-grade architecture for processing Shopify order events, following best practices throughout:
1. Entry Point: API Route Handler
import { NextApiRequest, NextApiResponse } from "next";
import { PrismaClient, Prisma } from "@prisma/client";
import { buffer } from "micro";
import { createHmac, timingSafeEqual } from "crypto";
import * as fs from "fs";
import * as path from "path";
import {
detectRecommendationType,
extractRecommendationSource,
formatLineItemProperties,
extractPlatterProperties
} from "../../../../utils/recommendation-utils";
import {
isEventProcessed,
markEventProcessed,
processOrderAsync,
logWebhookMetrics,
ShopifyOrder
} from "../../../../utils/webhook-utils";
- Uses Next.js API routes for seamless serverless architecture
- Imports specialized utility modules for core functionality
2. Raw Body Processing & Middleware Configuration
// Disable the default body parser to get the raw body
export const config = {
api: {
bodyParser: false,
},
};
- Disables automatic body parsing to access raw request data
- Essential for cryptographic signature verification
3. HMAC Verification for Security
function verifyWebhookSignature(
body: string,
hmacHeader: string | string[] | undefined
): boolean {
if (!hmacHeader || Array.isArray(hmacHeader)) {
return false;
}
const generatedHash = createHmac("sha256", SHOPIFY_API_SECRET)
.update(body, "utf8")
.digest("base64");
return timingSafeEqual(
Buffer.from(generatedHash),
Buffer.from(hmacHeader)
);
}
- Implements Shopify’s recommended HMAC-SHA256 verification
- Uses timing-safe comparison to prevent timing attacks
4. Main Handler with Error Boundary
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
const startTime = Date.now();
const topic = "orders/create";
let shopDomain = "";
let eventId = "";
let success = false;
// Only allow POST requests
if (req.method !== "POST") {
return res.status(405).json({ error: "Method not allowed" });
}
try {
// Processing logic...
} catch (error: unknown) {
console.error("Error handling order created webhook:", error instanceof Error ? error.message : error);
// Log metrics
logWebhookMetrics(topic, shopDomain, eventId, false, startTime);
// Even for errors, return 200 if we've parsed the body to prevent retries
// Only return 500 for critical errors that should be retried
return res.status(500).json({
error: "Internal server error",
message: error instanceof Error ? error.message : "Unknown error occurred",
});
}
}
- Complete error boundary with detailed logging
- Metrics tracking for all request outcomes
5. Idempotency through Deduplication
// Check if we've already processed this event
if (isEventProcessed(eventId)) {
console.log(`Event ${eventId} already processed, acknowledging receipt`);
logWebhookMetrics(topic, shopDomain, eventId, true, startTime);
return res.status(200).json({ success: true, status: "already_processed" });
}
- Prevents duplicate processing of the same event
- Essential for reliability with Shopify’s retry mechanism
6. Diagnostic Logging
// Debug: Write the webhook payload to a file for inspection
try {
const timestamp = new Date().toISOString().replace(/:/g, "-");
const logDir = "./logs";
// Create logs directory if it doesn't exist
if (!fs.existsSync(logDir)) {
fs.mkdirSync(logDir, { recursive: true });
}
// Write webhook details to file, but don't block the response
fs.promises.writeFile(
`${logDir}/specific-webhook-${timestamp}.json`,
JSON.stringify({
headers: req.headers,
body: order,
hmacVerified: true,
eventId
}, null, 2)
).catch(err => console.error("Error writing webhook log:", err));
} catch (logError) {
console.error("Error setting up webhook logging:", logError);
}
- Non-blocking payload logging for debugging
- Structured JSON format for easy analysis
7. Non-Blocking Processing Pattern
// IMPORTANT: Acknowledge receipt immediately with 200 status
// This tells Shopify we've received the webhook and prevents retries
success = true;
logWebhookMetrics(topic, shopDomain, eventId, success, startTime);
// Start processing the order asynchronously
setImmediate(() => {
processOrderAsync(order, shopDomain, eventId, prisma)
.catch(error => {
console.error(`Async processing error for order ${order.id}:`, error);
});
});
// Return success response immediately
return res.status(200).json({ success: true });
- Implements the crucial “acknowledge-then-process” pattern
- Uses Node.js
setImmediatefor optimal async execution
8. Intelligent Recommendation Analysis
export function detectRecommendationType(properties: Record<string, any>): string | null {
// Multi-pattern detection logic for recommendation types
}
export function extractRecommendationSource(properties: Record<string, any>): string | null {
// Advanced source extraction with fallback strategies
}
export function formatLineItemProperties(properties: any): Record<string, any> {
// Robust normalization of Shopify's property formats
}
- Sophisticated pattern matching for various recommendation formats
- Flexible property normalization for Shopify’s evolving data structures
9. Performance Monitoring
export function logWebhookMetrics(
topic: string,
shopDomain: string,
eventId: string,
success: boolean,
startTime: number
): void {
const processingTime = Date.now() - startTime;
console.log(`WEBHOOK_METRIC: ${topic}, shop: ${shopDomain}, eventId: ${eventId}, success: ${success}, time: ${processingTime}ms`);
}
- Detailed performance metrics with timing information
- Structured logging format for easy analysis and alerting