I have a read-only app tools extension (admin.app.tools.data, api_version 2026-04, CLI 4.3.0). It keeps getting blocked by the Sidekick safety scan after deploy, and the block notice gives no information at all about what failed - no field, no tool, no rule, no log. Just “didn’t pass our safety scan.”
I’ve now made three rounds of changes across three deploys, and each one is blocked the same way with zero specifics:
Version 1030221627393 - blocked. I made every tool read-only: the “draft” tools no longer mutate anything; they return a deep link to a pre-filled form in my app, and the merchant saves there. (Per the policy: data extensions read-only, mutations on the merchant’s confirmed save.)
Version 1030241091585 - still blocked. I reframed every tool and extension description away from any action/mutation wording (“prepare and return a link,” not “create/update”).
Version 1030250037249 - still blocked. I stripped all imperative / behavioral-override language (“NEVER,” “You MUST,” “Always,” “never guess”) from every tool description, input-schema description, and instructions.md, leaving plain factual descriptions of what each tool returns.
App ID: 340770258945.
Each deploy validates and bundles fine; the block arrives afterward as a separate notification.
Questions:
Is there any way to see the specific reason a version failed the safety scan - which tool/field/rule tripped it? Right now it’s a black box and we’re guessing blind.
Are there known triggers beyond mutations and imperative instructions that we should check? The guidelines page is high-level; we’ve addressed everything on it that we can identify.
Is there a way to get the scan result logs, or a contact for de-blocking review?
The repeated block with no actionable feedback is making this impossible to debug. Any detail on the failure reason would unblock us immediately.
Hi, Luke! Good news first: the version you deployed most recently appears to be passing the safety scan now, so you should be unblocked. Please let me know if you’re still hitting a block!
On the why: the block was coming from mutations in your data tools. The admin.app.tools.data target is for read-only data retrieval, but a few of your tools described state changes or resource creation (e.g. toggling/activating something, or staging a “draft” create). Those count as mutations even when the actual write happens later on the merchant’s Save, and that’s what kept tripping the scan.
The fix is to split data tools and actions:
Data extension (admin.app.tools.data) → read-only only: search, fetch, summarize. No create/update/delete/toggle/activate.
Action extension → anything that changes state, including staging a change for the merchant to confirm.
One tip to debug faster next time: while running shopify app dev, scan pass & fail results now shows up in your app’s Sidekick event logs in the dev dashboard (Apps → [Your app] → Logs section), so you can catch this before deploying. More in the docs at: Sidekick app extensions
Hey @abarrus, so yeah latest version was a rollback without the tools.
All I was trying to do is create a deeplink to help merchants create things in my app, which provided URL params to prefill form fields, it never created the resource, the deep link would simply direct the merchant to a page in my app with form fields prefilled and the save bar would be presented letting them either discard or save.
Funnily enough, I tried deploying the exact same thing again and its now passed and worked..
Also FYI, the logs are of no additional help, they literally give the exact same information the email says. So it would be nice to have more visibility if possible.
Following up - this is now actively blocking on what looks like scanner inconsistency.
Version 1030796410881 was blocked. The only change from my previously-PASSING version was: removed two read-only tools, added two read-only tools (one a near- identical clone of an existing list tool that already passed). No new mutations, no changed write tools, no new imperative language. Adding two read-only data tools should not flip a pass to a fail.
This matches what you saw earlier: I redeployed identical content and it passed. So the scan is returning different verdicts for equivalent (and in this case strictly-more-conservative) inputs.
Two concrete asks:
Is there ANY way to see which specific tool/field/rule tripped a given version? The dashboard logs repeat the email verbatim with zero specifics, so every round is a blind guess. Even a single “tool X, reason Y” line would unblock us instantly.
Can the scan verdict be made deterministic for identical input? Right now we can’t tell a real policy violation from a flaky re-roll.
Happy to share the full tool manifests for the passing vs blocked versions if that helps you reproduce.
Concrete proof of the non-determinism, with version IDs:
1030796410881 - BLOCKED.
Immediately redeployed identical content, no changes (version: stockful-dev-15) - PASSED, tools now serving fine.
Same manifest, opposite verdicts, minutes apart. So this isn’t something I can fix by editing descriptions - the input was identical.
One more question on top of the two above: does a blocked Sidekick extension hold the entire app version, or only gate its own Sidekick surface? i.e. if a flaky block lands on a release, does it also stall the other extensions/config bundled in that same deploy, or just remove Sidekick availability until a re-deploy passes? That determines how much risk a single flaky scan actually carries for us.
Hey Luke, thank you for the detailed version IDs, and for sticking with this.
On the non-determinism: we looked into the versions you flagged, and you are right. The blocked version 1030796410881 is byte-identical to the version that is live and serving right now, and the two came back with different results. This is not something you can fix by editing descriptions. The extension safety scan is an LLM-based check, and its detection is not always 100% consistent, especially in gray areas. That is a real gap on our side, and we are tracking it.
Separate from that, one thing is keeping you close to a block. The admin.app.tools.data target is read-only. But some of your tool names read like they change state, such as draft or toggle style tools. Those can count as mutations, even when the actual write happens later on the merchant’s Save. So the scanner can legitimately flag them. We recommend moving anything that changes state into an action extension, and keeping only read, search, and summarize tools on the data target. That is what gets you off the edge and passing consistently.
A block does not hold your app version, and it does not stall anything else in the deploy. The content scan is not part of shopify app deploy. Your version validates, bundles, and distributes normally, and the scan runs asynchronously. If it blocks, the only effect is that your app’s Sidekick tools stop being surfaced until a passing version lands. Every other extension and config in the same deploy ships and keeps working. So a single block only costs you Sidekick availability for that app.
In the meantime, if you hit a block on content you believe is clean, re-deploying the same version will usually clear a transient one. The most durable fix is still moving any state-changing tools off the data target.
Thank you again for the concrete repro. It is helping us confirm the determinism gap, and it is shaping how we prioritize the fixes.
I spent an hour debugging this today really unclear why my extensions stopped working into Sidekick until I followed the forum’s instructions to check logs and filter events by Sidekick. Would be amazing to have this an an email event in the future?
Good news first: the version you deployed most recently appears to be passing the safety scan now, so you should be unblocked. Please let me know if you’re still hitting a block!