Currently Shopify sends identifying information about the shopify store that a webhook payload is for, but it doesn’t send which Shopify Client ID or which webhook subscription caused the webhook to be delivered. This information is captured and displayed in the Delivery Logs in the partners dashboard, but it’s not sent in any headers.
If you have more than one Shopify application running on the same set of servers, this makes identifying which application an incoming payload is for harder! If you have an exact 1-1 mapping between infrastructure and client IDs, then you implicitly know which client ID you are, but if you have multiple, there’s no clear indicator in the payload. The reasons for having multiple client IDs around might be:
cost optimization, where you run multiple apps in the same process or against the same database just to save on costs for deploying more than one app
temporary migrations from one client ID to another
multitenant platforms like integrations providers or Gadget that host many Shopify apps
If you do have multiple client IDs, one way to identify which client ID caused a webhook to be sent is to check the HMAC of the payload against all the client secrets that could have possibly caused the webhook to be sent. Whichever client secret validates the HMAC is the one that sent it. This strategy is prohibitively expensive if you have many client IDs thought.
Another strategy for identifying which client sent a payload is to put the client ID in an HTTPS webhook subscription URL. This allows you to know which client sent a payload because you can create dynamic URL patterns in your application, and pass information back to yourself within that URL. I currently do this to work around this problem, with URLS like:
// for client id 1
"https://myapp.gadget.com/api/shopify/webhooks/1"
// for client id 2
"https://myapp.gadget.com/api/shopify/webhooks/2"
But, this strategy doesn’t work for the EventBridge / GCP PubSub subscriptions that Shopify recommends, because you can’t do dynamic topic patterns in these systems, just static topic names. GCP also has hard limits on the number of topics you can create total, and its expensive infrastructure-wise to subscribe to a lot of topics at the same time, so ideally we could put all our webhooks on the same topic, like:
all-webhook-payloads
and then inspect the individual payloads for which client id they are for.
Any chance we could get the client id added as a webhook header and pubsub metadata item?
I assume you’re asking for this functionality on behalf of Gadget’s infra for supporting partners using your platform.
For the normal use case, the app ID is an extra detail that’s implicit.
Just to play devil’s advocate, this scenario by a single Partner actually dances very close to Shopify App Store Guideline #6 - Overlapping Functionality.
I’m not saying this definitely is, but it could be interpreted that way.
If I were developing a platform for building Shopify apps on behalf Partners, I would personally just invest the time to build an ingress system based on Lambdas/Serverless Functions with a predictable URL structure that includes their app ID:
https://shopify.wh.${appId}.gadget.dev
Then you can map the incoming messages to the appropriate tenant in your platform.
Just a thought, that way you have more control rather than lobbying for this feature.
Just to play devil’s advocate, this scenario by a single Partner actually dances very close to Shopify App Store Guideline #6 - Overlapping Functionality.
We aren’t duplicating Shopify’s functionality – we’re using exactly the primitives they offer to help app developers build apps for their ecosystem. We’re not replacing other apps, or replacing Shopify, we’re a hosting provider that knows about Shopify webhooks and makes it easy to receive them.
Lambdas/Serverless Functions with a predictable URL structure that includes their app ID
This is exactly what we’ve done, as I said in the original post, but it is not the best practice. Shopify explicitly recommends using GCP PubSub, and I agree with their recommendation. Decoupling consumer and producer throughput is a big stability win, especially for BFCM, and PubSub is more efficient for both them and for us as you can use batch consumers and you don’t need to verify HMACs. We’d like to adopt the best practice for these reasons, but can’t because of the aforementioned limitations around topic cardinality in GCP or EventBridge.
So, if anyone from Shopify is listening, would you be open to adding that header?
Hi @airhorns this is something that we are aware of and have considered. When we looking at exposing the subscription ID it made sense for GraphQL (shop scoped) subscriptions, but less so for the TOML (app scoped) subscriptions as the identifiers were fairly meaningless. We’re considering other options such as the ability to include metadata on the webhook subscription that could be included at delivery time. At this point we have not yet committed to a solution and cannot provide any timelines, but it is something we’re looking at improving.
Ok, glad to hear this is on the radar! I see what you mean about subscription ID – I think Client ID wouldn’t suffer from the same strangeness though! I feel like it might make sense to just send both!
Hi