Any guidance how to handle login to Shopify for automated UI testing of Shopify app using Playwright, Cypress etc?
We wish to run this in our CI servers, but the frequent login confirmation pages or cloudflare captcha (even after first successful authentication) is causing flaky behavior).
How are you handling automated browser testing of your apps?
Unfortunately I don’t think there is a good way to do automated UI testing for embedded Shopify apps. We looked into this a while ago but ultimately decided to pass on trying to do it.
Like you mentioned, if you try to access the Shopify admin with an automated tool you’ll often get a Cloudflare captcha, especially if you’re trying to access the admin from a datacenter IP. You might be able to temporarily get around this by persisting the browser session, and manually logging in with that session to complete a captcha when it appears, but it’s not a stable solution.
Additionally I’m not sure if automatically accessing the Shopify admin, even to test an app, is permitted. Section 1.9 of the Shopify terms of service says that:
You agree not to access the Services […] using any robot, spider, scraper, or other automated means.
The only possible solution I could think of to get it to run reliably would be to make your own HTML page that loads up your app in an iframe. I don’t know if this is even possible though, you’d have to replicate a bunch of the app bridge and session authentication features.
I came across this problem before Unable to run automated test in Shopify app in embedded mode reliably · Issue #222 · Shopify/shopify-app-bridge · GitHub. Most devs just seem to login to playwright and run the test manually unfortunately there is no way to run this in the CI
We do need a way to mock app bridge reliably to run the E2E in CI
Looks like there is a PR to run Playwright test now Add End-to-End Testing Infrastructure with Playwright by IliasHad · Pull Request #986 · Shopify/shopify-app-template-remix · GitHub 
Don’t forget you’d also need to disable 2FA auth on your Shopify account, which is a security issue.
It might be possible to create a staff account with limited permissions to run these tests on behalf of. But using your main partner account for E2E tests would be an issue because of 2FA.
I have opened a feature request to make this functionality available from the Shopify CLI:
This PR is still open after 5 months with no reviews. 
Did you find any reliable solution?
Every once in a while I search if this is possible. Its annoying that with embedded app we have this limitation.
I believe there is a world where we can take the app and run it without app bridge outside of the Shopify iframe.
The issue is, with the additions features from the Shopify app bridge, its at a point the app won’t fully function without it if you use it.
My solution was to mock the AppBridge completely.
Here’s the NPM package: GitHub - ctrlaltdylan/mock-bridge: A complete mock of Shopify App Bridge to unlock automated browser testing, Chrome MCP and more for Shopify Apps
You can use npx @getverdict/mock-bridge http://localhost:3000 to launch your local app (running on localhost:3000) in a mocked Shopify admin session.
That way there is no auth, captchas or 2FA to hinder browser testing.
I also do something like this, my mock is pretty simple and follow the app bridge types.
The real problem for me at least is how to test theme app extensions since there are 10000 different themes with infinite variations.