The 500 ms 95th percentile response time target for carrier service callbacks, for apps that depend on carrier APIs

This is feedback on the Built for Shopify requirement that carrier service callbacks have a 95th percentile response time under 500 ms. It applies to any carrier service app that gets its rates from carrier APIs. We use our own app, built for Brazilian merchants, as the example.

  1. The network uses most of the budget before the app runs

Shopify calls the callback from US data centers and opens a new connection for every request. In our case each request pays:

  • about 150 ms between Shopify and the edge in the US, which we cannot see or control. It includes the connection setup
  • about 150 ms more from that edge to our server in Brazil, which is the round trip across the ocean

That is with Cloudflare Tunnel and Argo Smart Routing. About 1 request in 10 still takes a slower route and pays about 400 ms on the second leg, and Cloudflare support confirmed there is no setting to change it. Our app has a 95th percentile response time of 100 ms when it calls no carrier, and by our estimate Shopify would still measure more than 600 ms. Today Shopify measures about 770 ms.

  1. Moving the server next to Shopify does not solve it

Carrier APIs are hosted where the carriers operate. Ours answer in 50 ms to 1,400 ms from inside Brazil. From a US server every live quote crosses the ocean and back, so the delay moves from one leg to the other.

  1. The alternative is to stop calling carriers, and that is very large

To answer without a live quote, an app has to keep its own copy of each carrier’s price table. For our market that means:

  • about 5,500 cities, plus thousands of postal zones inside large cities
  • a dozen weight bands and up to 190 price levels per carrier
  • tens of thousands of quotes to fill one table
  • one table per merchant, because each negotiates its own prices
  • several carriers per merchant, out of more than a dozen in the market
  • a refill whenever a carrier changes prices, which happens several times a year without notice
  1. Even with that cache, the target is barely reachable

We estimated the 95th percentile response time with about 95% of requests answered from the cache, so that only 1 request in 20 still calls a carrier:

  • server in Brazil: about 625 ms, because the network alone keeps it above 600 ms
  • server in the US: about 495 ms

So the full cache plus a move to another continent lands just under the target, with no margin. These are estimates from our own request logs, not production results.

Suggestions

For an app in Brazil, the number Shopify measures today says more about the distance to the US than about the quality of the app. An app can do everything right and still miss the target, so the target does not tell good apps from bad ones in this market. We want to deliver better apps, and these changes would let the metric guide us:

  1. Keep a connection to the callback open and reuse it, so apps do not pay the network setup on every request.
  2. Show apps how the 95th percentile response time is built: which requests count, from which regions, and how much is connection time and how much is response time. Today we cannot see where the time goes on Shopify’s side.
  3. Measure from a region near the shop, or account for the distance between Shopify and the market the app serves.
  4. Publish guidance for carrier service apps that depend on carrier APIs, with what Shopify expects them to do to reach the target.
1 Like