We are creating automation test scripts for our Shopify apps. We wanted to confirm whether automation testing is possible on an unstable build. Currently, our development team is working on a new feature and has asked the QA team to create and run automation test cases. Is it possible to execute automation tests before the feature is deployed to the development environment? Additionally, if the feature is deployed to the development environment but has some functional issues, is it still possible to run the automation test cases successfully?
Depends on what you’re testing and the type of app you have.
Is it embedded in the Shopify admin?
Can you clarify what types of automation tests you’re attempting to do?
Do you mean unit testing code, or automating browsers through Puppeteer/Selenium to actually test the functionality through the users view?
We are automating browsers through Selenium with Java to actually test the functionality through the users perspective.
We are facing an issue with our automation flow. Our test funnel works like this: we open the site URL, search for a product, validate the assertions on the product page, and then move to the cart page. On the cart page, we apply assertions on the subtotal to verify whether the expected value matches the actual value. If the cart assertions pass, we proceed to checkout.
The problem comes when the feature is still under development or unstable. In such cases, we are unable to fetch the actual value from the site because, until the feature is fully developed, we cannot add rules in the admin. As a result, the rules do not reflect on the frontend, and we don’t get the actual value.
Sounds like you should make a new hosted environment for your app while its still in “staging”.
Then you could have a specific develoment store install your staged app and perform these customizations through the Shopify Admin and run your scripts.
Shopify just released the ability to allow specific crawlers as well, that should help with the CAPTCHA issue: Authorize custom crawlers and tools with Web Bot Auth - Shopify Changelog