E2E tests on storefront result in 429

We’re running E2E tests of our storefront solution using Cypress against a Shopify storefront. The tests essentially add one or more products to the cart and verify that the cart page and checkout page render as expected. The system under test is a combination of a script (App Embed), back-end APIs (exposed via proxy’s), the Cart.js API and the checkout (including discount functions).

This suite of tests represents a significant investment for us and allows us to ensure a quality product.

Until recently we were able to run our batch of tests (~500 scenarios) in roughly 40 minutes. Since July our tests have started failing (around 40 to 50 scenarios in) because of 429 errors (on Cart.js API requests, mainly add to cart) which then causes all subsequent tests to fail.

429 errors indicate that Shopify has detected that too many requests are issued by the same IP-address. This could potentially indicate that Shopify or Cloudflare have recently modified their bot detection mechanisms to be more aggressive.

Has anyone else run into this? What mechanisms are you using to run automated tests? Is there a way to white-list our traffic / store?

1 Like

Hey @BartCoppens :waving_hand: , the AJAX cart API has no hard rate limits on the number of requests, but it is still protected against malicious traffic via abuse-prevention measures as you mentioned there.

Generally, if requests appear suspicious (high volumes from a single IP), Shopify may respond with 429 or 430 errors to maintain platform stability. That said, you mentioned the failure rate seems to have become more frequent just as of recently? If you’re able to share a specific date when you noticed this first happening or an X-Request-ID from an API response header for an example call, I can track an example error in our logs and investigate further for you for sure.

I can’t guarantee that I’d be able to share any internal information on this at the moment, but I’m more than happy to investigate at the very least to confirm this is currently expected behaviour for sure.

Hi Alan, see Bug: 429 page errors are too aggressive - #25 by TPA_Admin

Hi @BartCoppens, thanks for sharing the link to the other thread there. I can’t share specific internal details currently, but I did just want to let you know that we are looking into this with our product team at the moment and the thread there with Kyle would be the best place for further details and assistance on this.

I’ll close out our thread here just for tracking, but please feel free to ping me in the other thread as well in case I can assist. I appreciate your patience while we investigated this, just wanted to loop back with you here to let you know there has been some movement internally on this.

1 Like