Replacing "storefront_digest" Cookie for crawling password-protected Shopify stores

Online Store and Theme Development
,
1

Short description of issue

storefront_digest seems not to be longer used. What should be used instead to crawl password-protected stores.

Reproduction steps

Based off : Using WebPageTest on Shopify preview sites with passwords ā€“ Performance @ Shopify

  • Open a password-protected store in a private tab
  • Check cookies, no storefront_digest cookie set
  • Enter the password and access the store
  • Check cookies, still no storefront_digest cookie set

Additional info

The storefront_digest cookies seems to have been merged inside the _shopify_essential cookie, which is opaque and not documented.

The cookie was used as a header in Screaming tool crawling tool to bypass the password page and crawl contents without removing the password (putting the store at risk of indexation / early discovery by bots). This was very useful in migration context and I wanted to know if there was a new method to achieve this behavior !

What type of topic is this

Troubleshooting

1 Like
2

I would also like to get some insight into this. Currently have a Shopify app which Iā€™m using for OAuth and allow site crawling with firecrawl. Right now I can only use the app on my development store which is password protected and I am unable to disable this. If I can understand how the password protection is working this would help figure out a workaround to crawl my shop for testing.