ProductVariants GraphQL query return empty results if the "first" amount is small

[soulchild]

I am using the same exact query, parameters, except with differing “first” amount.

x-request-id: 344489ca-1254-42b5-9c10-6b4c683663ba-1743531803
When the first is set to just 20 , no results are returned (but the pageInfo hasNextPage is true)

x-request-id : e8da295d-ef4a-4aa0-bb7c-3d077918830f-1743531880
When first is set to a higher amount like 200, results are returned, but it returned less than 200 result (about 150 ish), and the pageInfo hasNextPage is true

I would like request just 20 result at a time due to rate limit concern, but it seems like no result is returned if I use a small amount of “first” , is this output intended from Shopify? Is there any workaround for this?

[Kevin_Sieger]

Hey @soulchild,

can you share a specific API call and the corresponding response?

Kevin

[soulchild]

Hi Kevin,

This happened on one of my customer’s store, sorry I am not able to divulge the whole API response publicly (I wasn’t able to replicate on my own test store), but I will mention the API URL, and input queries.

The API endpoint is admin/api/2025-01/graphql.json

I am trying to loop through all productVariants that belongs to a collection, and I am using the endcursor from the pageInfo , to start the pagination for the subsequent calls (using the $vcursor variable)

query($vcursor: String) {

      productVariants(query: "collection:435226214617", after: $vcursor, sortKey: TITLE, first: 200){
        pageInfo {
          hasNextPage
          startCursor
          endCursor
        }
        edges {
          cursor
          node {
            id
            
          }
        }
      }

    }

I noticed that after several iteration (with first set to “20”), the “edges” started return empty, and the pageInfo startCursor and endCursor become null, but hasNextPage is true.

If I tried to set the “first” to 200, about 155 results are returned.

[KyleG-Shopify]

Hey @soulchild

Could you try removing the sortKey parameter completely and see if that resolves the pagination issue? Since you’re filtering by collection and sorting by TITLE, this mismatch might be causing the inconsistent results.

From the docs:

Sort the underlying list using a key. If your query is slow or returns an error, then try specifying a sort key that matches the field used in the search.

If removing the sort key doesn’t resolve the issue, this could be related to the product index in the store. In that case, please reach out to our support team directly as we have specific tools that can help resync the product index, which often clears up these types of inconsistencies.

[soulchild]

Hi @KyleG-Shopify ,

I have tried removing sortKey parameter, and the same issue still occur

first set to 10

first set to 100

In that case, please reach out to our support team directly as we have specific tools that can help resync the product index, which often clears up these types of inconsistencies.

Can you elaborate more on the “support team” ? Do you mean I should inform the merchant to contact Shopify support to resync the product index? Or I should contact partner support? (I dont think there’s any partner support channel other than this forum)

Is there a way for merchant to resync the product index themselves? I am not sure how many merchants are affected, and if there will be future affected merchant, I only knew this because one merchant informed me. I don’t have resources to check if all merchants using my app are affected and ask them to contact support one by one unfortunately .

[KyleG-Shopify]

Good question. When I’ve seen similar issues, its specific to a single shop not all merchants. Depending on your relationship with the merchant, you can have them reach out. It may be more effective to reach out through your partner account though, as you have more of the API Specific context (link this thread in your communication so they have the context as well).

https://help.shopify.com/en/support/partners

[soulchild]

This issue has been re-appearing on at least 3 merchants who are using my app, 2 of which has uninstalled my app due to this issue.

It seems like it is very common for a store to have product index cache issue? I was assured that this is a rare occurrence and can be resolved by asking merchant to contact support, but the product index cache reset takes 1-2 days to complete, and some merchants do not have the patience to wait (for the reset to complete, and also to wait for their turn to reach merchant support) and will just uninstall the app.

It feel very unfair for app developers to being penalized because of a Shopify internal issue

[KyleG-Shopify]

Hey! I’m happy to look in to this closer to see why it’s continually happening.

Since the issue is resolved by reindexing the cache, we should look at what’s happening earlier to get in to it’s current state. Does your app make any changes or updates to the products or variant’s prior to your query? Is your app making any similar queries before this with potentially slightly different query filters? Does it ever resolve itself over time? Ie, if you re-run the query after 24 hours is it still out of sync?

The reason I ask is to try and get an idea of the state of the products before the troublesome queries so we can get to the core of the issue.

[KyleG-Shopify]

Hey! Just checking in to see if you have any additional context to help us look in to this.

[soulchild]

Hi @KyleG-Shopify , thanks for checking in! Sorry for the late as I was swamped with my app operations recently.

For context, my app updates metafields for the variants that appears in the query, my app will query by a batch of 20 for product variants in a collection, and update metafields on them, then query the next batch, and update metafields and so on. (until there are no remaining product variants left to query)

Sometimes my app does make similar queries before this, the queries are either querying product (first 10) or productvariants based on given product IDs.

products(first: 10)

productVariants(query: "product_ids: .... ")

The issue does not usually resolve itself over time, I have set up notification on my own app, when I get notified, I will stop the querying background job manually, and notify merchants about this, and tell them to contact support to reset product cache.

if you re-run the query after 24 hours is it still out of sync?

So far I haven’t able to test this yet, as most merchants will uninstall my app before the product cache is successfully reset.

[KyleG-Shopify]

Thanks for sharing those specifics. I’m digging in some more to see what I can find.

In the meantime, thinking of other workarounds, have you considered using bulk operations for your queries and mutations? Especially since you mentioned earlier being aware of rate limits this may be a more efficient way to do this.

If that’s not viable, try adding reverse:true to your queries to see if that reduces the number of occurrences. That will sort from most newest to oldest which can help.

Another option for when you get an empty array but there’s a next page, would be to increase the request to 100 or maybe even higher up to 250 to ensure you’re at least getting the next cursor so you can continue to paginate.

[Chris_Geelhoed]

@KyleG-Shopify It is really nice to have Shopify engineers like yourself helping developers like us with issues with the productVariants API, but instead of brainstorming untested one off ideas which may or may be applicable, and may or may not even help, it may be more helpful for Shopify to reflect on the overall reliability of the endpoint, advocate for long term changes to make the API more reliable for developers, and transparently help us understand why the endpoint is so error prone and what is being done to make it more reliable.

Especially with the variant increase from 100 to 2048 looming, I think this issue needs to be recognized as important and is deserving of substantial engineering effort from Shopify.

Again, any help at all is absolutely welcomed so I’m torn writing a message like this, but after facing issues with the endpoint for years developers like myself have heard all the “Hey have you tried this?” workarounds. At some point the API needs to work.

[KyleG-Shopify]

Hey Chris. I really appreciate your response. Don’t feel torn writing a message like you have, as all feedback helps and we do pass it on.

For context, I was leaning in to trying to be helpful by looking in to available workarounds I have seen work before, but the suggestions are not intended to be long term fixes. I will continue to look in to this.