Operations Performed with Offline Tokens Are Not Reflected in API Responses When Using Online Tokens

[Shota_F]

• Inconsistency in Rate Limit Points
  • When consuming a large number of API points using an offline token, the value of extensions.cost.throttleStatus.currentlyAvailable retrieved with an online token does not reflect this consumption.
  • In contrast, when API points are consumed with an online token, the consumption is accurately reflected when checking with the same online token.
• currentBulkOperation Returns Null
  • After executing bulkOperationRunMutation with an offline token, retrieving currentBulkOperation(type: MUTATION) with an online token returns null.
  • However, if bulkOperationRunMutation is executed with an online token, retrieving currentBulkOperation(type: MUTATION) with the same online token returns the correct result.
• Common Points
  • In both cases, operations performed with an offline token are not reflected in API responses when using an online token.

How to Confirm This Issue

1. In the background, use an offline token to execute GraphQL queries that consume more than 90% of the maximumAvailable points (exceeding the restoreRate).
2. Prepare a server that uses an online token. From the app screen, send an appropriate query to this server once per second and check the value of currentlyAvailable returned in the response.

We can check currentBulkOperation in the same way.

[Alan_G]

Hey @Shota_F - thanks for flagging this! It does seem odd, as we’d generally expect rate limits and statuses to be reflected accurately across token types.

For the rate limit issue, could you share an X-Request-ID and a timestamp if possible from an example offline call consuming more of those points than expected, and the X-Request-ID/timestamp from the subsequent query that didn’t show the updated currentlyAvailable count?

For the bulk operation issue, could you also share the gid (e.g., gid://shopify/BulkOperation/...) of an operation started offline, plus the X-Request-ID and timestamp for the specific query call to currentBulkOperation that returned null?

I can use that info to track down what might be happening on our end here as well as to try and replicate the issue. Hope to hear from you soon!

[Shota_F]

I thought it would be easier to understand if you could see a demo, so I’ve attached the code and a video.

GitHub - shota-f/shopify-app-token-types-demo

Overview of this demo

• Uses Shopify’s official Remix app template
  • Passes useOfflineTokens: true during initialization
• Performs high-cost updates on 190 products using the regular API rather than bulk operations (190 × 10 = 1900pt)
  • The update adds the current time to each product title
  • Implements both online and offline endpoints
  • For simplicity, the offline endpoint is implemented on the same Remix server, though in reality it would be on a separate server
• Retrieves and displays throttleStatus once per second
  • Implements both online and offline endpoints
• Created a new app and development store to avoid any special side effects

Issue: Tokens do not affect each other’s throttleStatus.currentlyAvailable

Consuming points with Token A does not affect Token B’s throttleStatus.currentlyAvailable. This appears to be the case for both online and offline tokens.

Issue: Switching tokens allows you to use at least twice as many points (unexpectedly nice)

You can perform the same high-cost operation with Token B before Token A’s points have fully recovered. I hope this doesn’t get fixed…

About BulkOperation

Due to time constraints, I haven’t created a similar demo, but after running bulkOperationRunMutation with an offline token, I’ve confirmed two things:

1. currentBulkOperation(type: MUTATION) using an online token does not reflect the operation
2. You cannot run bulkOperationRunMutation simultaneously with an online token

[Alan_G]

Thanks for the details here @Shota_F - I’m doing a bit more digging into this on my end to confirm expected behaviour, but I’ll loop back with you when I have more info

[Alan_G]

Hey again @Shota_F - was able to get some confirmation on this for you. Currently, online and offline tokens don’t share a rate limit pool, so that’s why we’re seeing the different point value showing between tokens, even though they’re related to the same app.

Similar logic is happening when it comes to the bulk operation status query. An online token can often have different access permissions than the offline token for the same app, so we do “attach” bulk operation queries to the token that initially created the bulk op, if that makes sense.

In terms of “swapping out” the tokens to double the API rate limit, after speaking with the team, this may be considered a breach of our Terms of Service and we would reserve the right to shut down an app that attempts this, so I did just want to recommend against using that workaround.

Also, I did want to confirm that there are no repercussions for your app at this time if you are using that method to boost the rate limit, but I did want to share that info regarding the TOS for clarity.

I definitely understand it’s not the most ideal answer, but hopefully this helps a bit/answers your questions - let me know if I can clarify anything here!

[Shota_F]

Hi @Alan_G, thank you for your research and explanation.

Regarding the fact that “points” and “bulk operation information” are not shared between tokens, I believe they should be shared, for the following reasons:

1. There are cases where we want to use point information, such as determining which query to execute based on the result of currentlyAvailable.
2. Sometimes, we want to display the status of bulk operations performed with an offline token in the UI of an app that uses an online token.
3. As a solution for 2, it is possible to use the offline token via the server from the online side, but the reverse is possible yet would result in awkward code.
4. Most importantly, sharing this information matches the specifications described in the documentation.

To elaborate on point 1, my various attempts to use the point information led to the discovery of the current behavior.
Initially, I felt that the period from the start to the end of bulkOperationRunMutation was slow.
After some trial and error, I thought, “Wouldn’t it be faster to repeatedly perform normal updates, using as many points as possible within the range that doesn’t affect normal app operations?”
The code looks like this:

javascript

// Use the offline token on the background server
const runMutation = async (args) => {
  const { requestsCount } = args

  // Get the currently available points
  const currentlyAvailable = await getCurrentlyAvailable(...)

  const COST_PER_REQUEST = 10
  const APP_REQUIRED_POINT = 100
  // If enough points remain after this operation, execute all requests in parallel
  if (currentlyAvailable - requestsCount * COST_PER_REQUEST >= APP_REQUIRED_POINT) {
    await Promise.all(...)
  } else {
    // Otherwise, fall back to bulk mutation
    await runBulkMutation(...)
  }
}

With this method, if the number of rows to update is below a certain threshold, the update is significantly faster and you can utilize available points without waste.

While experimenting with this approach in both the app and the background server, I discovered the two issues mentioned earlier.

Since it is known that “points are shared between the app and the shop,” I have not performed “double execution by token swap” (I only discovered this issue while writing code to explain the problem).

However, depending on the app’s design, there is a possibility of “unintentional double execution” occurring.

To prevent such issues, I believe it is reasonable to (1) make the information obtainable with each token the same, and (2) make any execution that uses more than the app’s currentlyAvailable fail by the system.

[Alan_G]

Hey @Shota_F,

Thanks for the follow up here - I really appreciate your detailed points in your reply there, you do raise some valid ideas.

For conditional logic, I can definitely see how having shared visibility into rate limits and bulk operation status across online and offline tokens would be beneficial like you mentioned, so I’m going to set up a feature request report on my end to pass this along to the team for their consideration as we plan future improvements. I can’t guarantee anything in terms of if/when this would be implemented, but I do see the pros of your ideas here for sure.

Again, really appreciate you sharing your insight!

[Shota_F]

Hi @Alan_G.

Thank you so much for handling this issue so carefully.
I hope that things will improve in a positive way