## Request behavior
- The crawler serializes Shopify requests with a maximum concurrency of 1,
waits 3 to 6 seconds between starts, and stops or defers work after a 429.
- Replacing an earlier OCI ephemeral public IP with another Oracle address did
not resolve the behavior.
- The production host now uses a reserved OCI address so its identity can
remain stable during investigation.
- Non-Shopify requests from the OCI instance continue to work normally.
- In the July 28 OCI run, three distinct Shopify storefront hosts directly
returned 429. The Shopify cohort ended with 0 of 28 source configurations
successful because the circuit breaker suppressed or deferred the remaining
work.
- A residential run using the same crawler identity and signing key completed
all 28 Shopify source configurations.
## Web Bot Auth validation
This uses Shopify’s third-party bot-operator flow, not a merchant-generated,
domain-scoped Crawler Access signature from Shopify Admin.
The live public directory returns HTTP 200 with the expected media type and a
signed Ed25519 directory response. I independently verified a captured
production request against the public key served by that directory. The
request and directory use the same key pair.
This establishes protocol-level cryptographic validity. It does not prove that
Shopify’s edge recognized the request as signed or indicate which trust or
enforcement tier Shopify assigned internally.
Operator information: < PurveyorsBot - Crawler Identity and Operator Policy >
Shopify higher-access form submitted: July 24, 2026. No reference number was
received.
## Paired comparison
### OCI production origin
```text
Source network: Oracle Cloud Infrastructure
Source public IP: available privately
Timestamp: 2026-07-28T01:06:22Z
URL: https://coffeebeancorral.com/collections/all-green-coffee-beans/products.json?limit=250&page=1
Method: GET
Crawler revision: f12adbb0ae8813c7da39da0456e3e06123d79e97
User-Agent: PurveyorsBot/1.0 (+https://www.purveyors.io)
Web Bot Auth: enabled
Signature-Agent: https://api.purveyors.io
Signature key ID: umx-V_Vvxhp0X0mwV7YErAGc_Q5BXgMBhwtf6ogq47c
Covered components: @authority and signature-agent
Resul@authority: HTTP 429
x-request-id: not retained
cf-ray: not retained
retry-after: not retained
Response excerpt: curl: (22) The requested URL returned error: 429
Time since previous signed OCI Shopify request: approximately 17h 48m
```
### Residential origin
```text
Source network: residential ISP
Run timestamp: active by 2026-07-26T19:09:12Z
Comparable source: Coffee Bean Corral
Configured URL: coffeebeancorral .com/collections/all-green-coffee-beans/products.json?limit=250&page=1
Method: GET
Crawler command: npm run scrape-local all
Crawler revision: not retained in the supplied transcript
Web Bot Auth identity: confirmed in the same run
Signature-Agent: api.purveyors .io
Signature key ID: umx-V_Vvxhp0X0mwV7YErAGc_Q5BXgMBhwtf6ogq47c
User-Agent: PurveyorsBot/1.0 (+https: //www. purveyors .io)
Coffee Bean Corral result: source completed successfully, 56 products found
Shopify cohort result: 28/28 sources successful, 842 products found
Individual HTTP success status: not retained
x-request-id / cf-ray / retry-after: not retained
```
The residential transcript does not retain its git revision or individual
successful response headers, so I am describing this as the same crawler and
signing identity rather than an identical byte-for-byte request.
Two additional Shopify storefronts returned HTTP 429 from the same OCI run:
- `2026-07-28T01:06:06Z`:
`https:// coffeegreenbeans .com/collections/greencoffeestore/products.json?limit=250&page=1`
- `2026-07-28T01:06:27Z`:
`https:// shop.thetaridgecoffee .com/products.json?limit=250&page=1`
## Questions for Shopify
- Was the captured request recognized as Web Bot Auth traffic for
`Signature-Agent: https:// api.purveyors .io and key ID \umx-V_Vvxhp0X0mwV7YErAGc_Q5BXgMBhwtf6ogq47c`? If not, which validation
step failed?
- Can Shopify determine whether these responses came from storefront rate
limiting, bot detection, IP or network reputation, Web Bot Auth validation,
or another abuse-prevention rule?
- Why does the effective permitted request rate from this OCI origin appear
to be zero, including after an extended idle period?
- Does Shopify apply different admission rules to cloud or datacenter
networks than to residential origins for public storefront crawling?
- Is the higher-access form the correct escalation route for this behavior,
or is there a separate process for investigating persistent storefront
admission decisions?
- Is there a supported process for registering one stable crawler egress IP?
- Which response identifiers and headers should I retain in a fresh bounded
canary so Shopify can trace the decision internally?
I can provide the current and previous OCI addresses, affected storefronts,
sanitized request headers, and a fresh request capture privately.
I am not trying to rotate addresses or bypass enforcement. I am trying to
operate through one stable, correctly authenticated crawler identity and stay
within Shopify’s supported usage model.