I would greatly appreciate someone looking into this.
This is crazy and infuriating!
Our entire e2e Playwright test suite is failing since 2 weeks!
Every single cart/add.js call goes in 429 statuscode making our entire suite useless.
Fix this asap!
1 Like
@KyleG-Shopify Thank you for your response.
We experienced the same issue as everyone above. Our store is covered with AQA tests, and they started failing massively. It’s a huge threat for a 100M+ store that relies on AQA tests for the release process.
If you could provide a bit more information of how to approach the AQA tests now - I’ll appreciate it a lot (same as everyone else).
1 Like
Thank you all for the detailed reports. That’s really useful to help demonstrate the impact of the errors you are getting. I’m continuing to look in to this for you and I am sharing what you have shared here with our storefront teams. I will update you as soon as I know more.
@KyleG-Shopify Escalating via Twitter since it’s been three weeks and you guys are clearly not taking this seriously. https://x.com/discountninjaio/status/1946201612302557287
3 Likes
To whoever is evaluating this ticket on Shopify’s / Cloudflare’s end: I understand the need for Shopify to address bot traffic and I understand Cloudflare’s position against AI crawlers (Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model | Cloudflare). However, please realize that app developers and large merchants have made considerable investments in E2E/QA tests and rely on them to release a properly tested product. Surely this is a concern Shopify faces internally as well? It would be appreciated if Shopify can work out a system where the E2E/QA test traffic would be whitelisted. It would also be appreciated that this matter gets urgent attention for all of the reasons @alexm @Andrea_Velasca @TPA_Admin and others have pointed out.
3 Likes
Our team is significantly impacted. We rely on these simple tests for our CI/CD system. We had an urgent need for a deploy on Friday. It took multiple logins throughout the weekend to coax our CI/CD system past these 429 errors to get a deploy to production. This used to take less than 1 hour.
Something changed. There is an improvement. I did get a few 429s, but I was able to run our CI/CD pipelines several times with only a few failures, which resolved with a single retry. This is far better than the condition we were in on Friday.
@TPA_Admin I confirm, something seems to have been improved. We were able to run a much larger set of tests today.
Hi @BartCoppens
Our team have been making updates to improve this experience.
2 Likes
Hello, our e2e playwright tests have severely been impacted by 429 and cloud flare validators within the last few weeks. Prior to this we ran checkout test just fine. We are not botting or hammering the end point , maybe running 50 automated tests in the span of an hour.
This problem is still persistent till this evening. We need a resolution that allows us to test our store front without being constantly treated like a bot.
It seems the 429 errors have died down but are now getting blocked by cloudflare validation screens when going to the checkout endpoint.
This is happened on both my local machine at home and our cloud gateway corporate machine
Please note that 429s are still happening. A retry generally works. But, our tests that are loading about 15 checkout pages have 1-2 failures on every cycle. A cycle takes 10 minutes and occurs maybe 1-2 times per hour. We are not abusing the system.
Thank you all for continuing to update us on the friction you are still seeing. This is a priority for our team and more improvements are coming.