I would greatly appreciate someone looking into this.
This is crazy and infuriating!
Our entire e2e Playwright test suite is failing since 2 weeks!
Every single cart/add.js call goes in 429 statuscode making our entire suite useless.
Fix this asap!
@KyleG-Shopify Thank you for your response.
We experienced the same issue as everyone above. Our store is covered with AQA tests, and they started failing massively. It’s a huge threat for a 100M+ store that relies on AQA tests for the release process.
If you could provide a bit more information of how to approach the AQA tests now - I’ll appreciate it a lot (same as everyone else).
Thank you all for the detailed reports. That’s really useful to help demonstrate the impact of the errors you are getting. I’m continuing to look in to this for you and I am sharing what you have shared here with our storefront teams. I will update you as soon as I know more.
@KyleG-Shopify Escalating via Twitter since it’s been three weeks and you guys are clearly not taking this seriously. https://x.com/discountninjaio/status/1946201612302557287
To whoever is evaluating this ticket on Shopify’s / Cloudflare’s end: I understand the need for Shopify to address bot traffic and I understand Cloudflare’s position against AI crawlers (Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model | Cloudflare). However, please realize that app developers and large merchants have made considerable investments in E2E/QA tests and rely on them to release a properly tested product. Surely this is a concern Shopify faces internally as well? It would be appreciated if Shopify can work out a system where the E2E/QA test traffic would be whitelisted. It would also be appreciated that this matter gets urgent attention for all of the reasons @alexm @Andrea_Velasca @TPA_Admin and others have pointed out.
Our team is significantly impacted. We rely on these simple tests for our CI/CD system. We had an urgent need for a deploy on Friday. It took multiple logins throughout the weekend to coax our CI/CD system past these 429 errors to get a deploy to production. This used to take less than 1 hour.
Something changed. There is an improvement. I did get a few 429s, but I was able to run our CI/CD pipelines several times with only a few failures, which resolved with a single retry. This is far better than the condition we were in on Friday.
@TPA_Admin I confirm, something seems to have been improved. We were able to run a much larger set of tests today.
Hi @BartCoppens
Our team have been making updates to improve this experience.
Hello, our e2e playwright tests have severely been impacted by 429 and cloud flare validators within the last few weeks. Prior to this we ran checkout test just fine. We are not botting or hammering the end point , maybe running 50 automated tests in the span of an hour.
This problem is still persistent till this evening. We need a resolution that allows us to test our store front without being constantly treated like a bot.
It seems the 429 errors have died down but are now getting blocked by cloudflare validation screens when going to the checkout endpoint.
This is happened on both my local machine at home and our cloud gateway corporate machine
Please note that 429s are still happening. A retry generally works. But, our tests that are loading about 15 checkout pages have 1-2 failures on every cycle. A cycle takes 10 minutes and occurs maybe 1-2 times per hour. We are not abusing the system.
Thank you all for continuing to update us on the friction you are still seeing. This is a priority for our team and more improvements are coming.
Will you PLEASE fix this issue?
Everything is coming back as 429 when we try to ping or scrape pages. Its really important that we aren’t blocked as partners. There are many reasons its needed.
WHITELIST our servers if you have to. PLEASE
Hey everybody, I dont know if this helps you, but i think it can be solved.
I have the same issue with using GUZZLE, and I reuse the same client, and i get mostly 429 and some 200, and it randomly changes from minute to minute.
However if i set up 2 separate functions that open and close the connections, it seems to work fine (always able get the right status and contents).
Heres my boilerplate code:
function get_contents($url, $allow_errors = false){
if ( empty($url) ) {
return '';
}
if ( !validateUrl($url) ) {
errorLog("Problem validating a url (get_contents): " . $url);
return '';
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,3);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS,3);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0');
curl_setopt($ch, CURLOPT_REFERER, 'https://www.google.com');
......
function remoteFileExists($url, $allow_429 = false) {
$ch = curl_init($url);
$options = [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HEADER => false,
CURLOPT_NOBODY => true,
CURLOPT_CONNECTTIMEOUT => 2,
CURLOPT_FAILONERROR => 0,
CURLOPT_TIMEOUT => 8,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_MAXREDIRS => 3,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36',
CURLOPT_REFERER => 'https://www.google.com'
];
curl_setopt_array($ch, $options);
curl_exec($ch);
$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$redirected_url = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
curl_close($ch);
....
It might be the curl settings, or the fact im closing and reopening the connections. NOT SURE, but I do not have time to debug this. feels like minimal viable pathway to making it work. Good Luck.
Ignore my last message. This does not work when you process many in a list - they just all start failing regardless. Has anyone tried swapping the USER_AGENTS and REFERERS to different values for each call? Will try that
DO THE SHOPIFY DEVS UNDERSTAND HOW IMPORTANT THIS IS?
For example, one of our tools checks if REDIRECTS still point to an existing page - If its 400, then we DELETE the redirect. The Fact that this is totally broken can be devastating for merchants!
Kyle, I wanted to let you know what the issue is persisting. It is not debilitating like before. We are running just a few tests and about 10-15 percent fail and need to be retried. That is 2 out of 15! We are only running 15 orders through these tests. That should not be flagged as abusive to Shopify.
@KyleG-Shopify Bumping this up too.
Our team is not able to perform the releases normally for weeks.
Recently, all permalinks on our website started acting up (something on the Shopify side, since we haven’t made any changes to these permalinks) driving our CVR down, and we weren’t able to even detect it timely, since there were no AQA tests running.
Hello, our team is still experiencing cloudflare 429s running checout automations
This morning we ran 4 tests in the span of about 10 minutes and the 5th one triggered the cloudflare captcha. We need a way to whitelist automation that’s being done on our testing environment.
I am replying again after another 6 days to say that it is still an issue.
We are running just a few tests and about 10-15 percent fail and need to be retried. That is 2 out of 15! We are only running 15 orders through these tests. That should not be flagged as abusive to Shopify.