Our app is responsible for delivering shipping rate calculations based on defined conditions — as described in the App Store:
Your shipping, your rules! Offer precise shipping rates tailored to every customer and destination. Create your own custom shipping zones by postal codes, cities or countries — perfect for remote or distant areas with higher delivery costs. Configure advanced shipping rate rules based on weight, order value, item quantity, product tags, time of the day & more. Add multiple shipping methods per zone, hide them in specific conditions & ensure accurate delivery prices for every location you serve.
Problem description
Recently, we’ve started receiving reports from merchants that in their admin panel they see a message stating that backup rates have been used (see screenshot).
We always review these issues thoroughly, yet we are unable to find any cause for the error:
In the past week we did not experience any server overload or infrastructure issues.
There are no errors in our application logs (backend: PHP 8.3, Symfony 7.3).
Looking at the server’s access.log, we see Shopify makes requests to our Carrier Service URL, and all of them return HTTP status 200.
According to Shopify Partners logs, our Success Rate is 100% — though there is a slight discrepancy:
According to our access.log (on October 5): 32,846 requests
According to Shopify’s partner dashboard: 32,840 requests (a difference of 6 requests)
Additionally, we tested customer flows manually: adding the same items to the cart, entering the same checkout data — we always receive the correct shipping method in the response.
Question
Has anyone else encountered a similar issue?
Could the Shopify team check on their side what might be causing the backup rates message to appear even though our Carrier Service endpoint is responding correctly (200 OK)?
Sometime Shopify does not call our API to get the rates and use the backup rates. Checking the log I see similar discrepancies, roughly 1 in 15k requests.
Hey folks - thanks for flagging this and for checking your infrastructure on your end there.
There are a few reasons why A 200 OK by itself could still potentially result in “missing” rates, since the response has to arrive before a timeout on our end and still include at least one usable rate for each shipment. Intermittent network issues or client/browser aborts could potentially result in this even if your side is returning a 200 as well. Returning zero rates due to strict rule conditions or payload validation issues like currency mismatches or delivery date formatting can also trigger fallback.
If you folks are open to sharing a couple of affected order IDs and any request IDs/timestamps for the requests Shopify sent sent out, I am definitely more than happy to see if we can review the backup rates history and help correlate timestamps with your logs to pinpoint whether this was a timeout, or another error. Hope to hear from you folks soon and hope this helps a little bit as well.
Unfortunately, we don’t currently maintain logs that are that detailed — we’ll need to implement that and are already discussing this internally.
For now, I can provide a few order IDs — hopefully that’s enough to start with:
6933509898573
6933266170189
6932083999053
6932027244877
You mentioned response times from our server — based on data from Shopify Partners: the 95th percentile response time is ~334 ms. I’m not sure what the numbers look like across all requests.
Hey @sebastian.pisula, Thanks for those order #s and no worries regarding the logging.
If possible, could you share rough timestamps (in UTC if possible) for when the backup rates showed and when the shipping step was attempted? Also, if you have your app ID handy, I think we’d have all of the info I’d need on my end to investigate this on our end a bit more deeply. Hope to hear from you soon, thanks for your patience on this as well, really appreciated.
Regarding the approximate timestamps, I believe the created_at field from those orders should be the right reference — I assume it corresponds to the moment when the customer attempted to go through the checkout.
With the order IDs, you should be able to see the exact dates and times.
App ID: 157557948417
Let me know if you need any additional information.
By the way — I recently migrated the webhooks in our apps to the App-specific version, and I’ve been reviewing the logs in the dev dashboard. I have to say, it’s a great tool!
Having similar monitoring for Carrier Service requests, like the one available for webhooks, would be extremely helpful for analysis and debugging.
Information such as request ID, response code, related order ID / checkout ID, or whether backup rates were applied would be very valuable for diagnosing and resolving such cases.
In the meantime, I wanted to implement improved logging on our side to record the Request ID, but it seems that the headers we receive don’t include anything like X-Shopify-Request-Id.
Here’s an example of the headers sent to our Carrier Service endpoint:
Thanks again for the order IDs and the app ID, that should give me enough to dig into our logs on my side. I’ll use the orders’ created_at timestamps as anchors and see what triggered the backup rates.
I also double-checked on our end here and can confirm the X‑Shopify‑Request‑Id header isn’t currently included on Carrier Service requests. Sorry for the lack of clarity there. I’ll pull the relevant traces using the headers you shared and loop back once I have more concrete info.
I’ll also forward your request to include a stable request ID in those calls going forward. I agree it would make correlation much easier. I can’t promise anything, but I’ll get it in front of the right folks. Appreciate your patience, I’ll be in touch as soon as I have next steps on my end here
Hey @Alan_G , here is an order that received a backup rate today 8157295739186. We returned the response 1345ms which should not qualify for a timeout. Our app id is 16570220545
Hey @Francois_Rulliere - thanks for sharing that. I did a bit of digging in our logs, and it looks like that order is from August. Can you confirm that’s the right order ID? Happy to dig into this further, just wanted to make sure I was looking at the right data on my end here. Thanks!
Hey @sebastian.pisula - thanks for the ping here and for following up. We’re still looking into things on our end, but I’ll update you as soon as I have more info on my end for you.
Thanks for sharing that @Francois_Rulliere - we should have that one in our logs, I appreciate you sending that our way. Just wanted to confirm that we’re still looking into this. I’ll reach back out once I have more info to share.
Hi @Alan_G we’re seeing an uptick of backup rates recently too, but checking our servers, there’s been no change in load or issues from what i can see, it’ll be good if i can get some insight into what shopify sees on their end, any error codes etc. In some instances we don’t receive the request at all, other times our records show that we have responded quickly to the request (<0.01s) yet backup rates still trigger, we’ve also had an instance where the merchant was making a draft order, the rates from the app showed, they selected the rate, then when they created the order the rate was replaced with a backup rate - in this case, if the merchant has already selected the rate, why is a backup rate showing?
@Alan_G Are you able to provide an update here please? I’m getting more reports of these from merchants, most of the orders we have matching logs on our end which shows our app returned rates without any timeouts, other orders we don’t receive any payloads. It would be good to get to the bottom of this before black friday sales. We’ve had significant trouble escalating these tickets via shopify support as the level 1 staff do not understand how shopify API works and refuse to escalate to technical support.
- thanks for flagging this and for checking your infrastructure on your end there.