Sidekick app extension: tools registered correctly but Sidekick returns "Skill not found" — intermittent across chat sessions

Extensions
1

Symptom: Sidekick reports our deployed app’s skill as “not found” when trying to invoke any of our 7 tools — but the behavior is intermittent. In the same dev store, same active app version, same browser session, two back-to-back Sidekick chats can produce different outcomes: one successfully invokes our tools and returns real data, the next responds “the skill isn’t available to me in this session.”

App identification:

  • App handle: weaver
  • App ID: c64a35d9e6e89ce367704b9cf1add9e4
  • Org: 0xlab
  • Active version: weaver-13 (deployed 2026-05-21)
  • Last version where Sidekick reliably invoked our tools: weaver-7 (2026-05-20 09:07:58 UTC)
  • Dev stores tested: weaver-dev-sy57ffgo.myshopify.com and a brand-new dev store created today — same failure on both.

Extension setup (current, all required fields present and well-formed):

  • Type: ui_extension
  • Target: admin.app.tools.data
  • API version: 2026-04
  • [sidekick] extensions_summary in shopify.app.toml: present, scoped only to currently-deployed connectors (Klaviyo, Notion, Google Sheets)
  • Extension description in shopify.extension.toml: present, ~250 chars, lists explicit “Use when the merchant asks about X” trigger phrases
  • tools.json: 7 tools registered, all with non-empty inputSchema.properties
  • instructions.md: per-tool invocation guidance for Klaviyo, Notion, Sheets

Key evidence — Sidekick’s own diagnostic response:

When I asked Sidekick to invoke tools by ID directly, its response was:

“every attempt to activate the Weaver skill has returned ‘Skill not found’”

“The root issue is that Weaver’s skill isn’t registering on my end, even though the app is installed.”

So Sidekick itself is telling us its backend returns “Skill not found” for our skill ID. This explains the intermittent behavior — when Sidekick’s cache serves a “registered” state, our tools work; when it serves the “not found” state, they don’t.

What we’ve tried (no effect):

  1. Uninstalled and reinstalled the app on the dev store
  2. Created a brand-new dev store and installed from scratch — same failure
  3. Rolled back to multiple previous app versions (weaver-7, 9, 10, 11) that previously worked — same failure
  4. Multiple redeploys via shopify app deploy; each succeeds with Included './tools.json' and Included './instructions.md'
  5. Verified shopify app info shows the extension as deployed, not draft
  6. Browser DevTools console: when tools work, [weaver] calling backend logs appear; when they don’t work, no fetch is even attempted — confirms the tool isn’t registered for that chat session

Suspected root cause:

For a brief window we had a deploy with inputSchema: { type: "object", properties: {} } on one tool (empty properties object), and our extensions_summary advertised connectors (Slack, Gorgias, Meta Ads) that weren’t actually in tools.json. Both have since been fixed in multiple subsequent deploys. We suspect Shopify’s Sidekick skill registry cached our app as “broken” during that window, and the fix is propagating unevenly — explaining the intermittent recovery across sessions.

Can the Sidekick team force a re-registration / cache invalidation for app c64a35d9e6e89ce367704b9cf1add9e4? Or confirm whether there’s a known propagation delay for tool registry updates that we should just wait out?

Happy to provide more logs, request IDs, or anything else useful.

2

Hi @mhynes

I believe you’re experiencing this because sidekick app extensions are not actually publicly available yet. Although the docs are live, the extensions are only currently accessible to partners in a closed beta program.

3

Hi again @mhynes

You may be seeing these error if you’re testing on a production/ non-dev store - if you try on a dev store created from the dev dash, these skills should be registered by Sidekick.

4

Thanks Liam — we are on a dev store created via the dev dashboard (weaver-dev-sy57ffgo.myshopify.com), and Sidekick was successfully invoking our tools as recently as two days ago (2026-05-20) on that same store. So the closed-beta gating isn’t the blocker for us.

The mystery is the intermittent behavior since then. Concrete observations within the last ~24 hours, all on the same dev-dash dev store and same active app version:

Chat session A: I ask “what are my Klaviyo flows?”, Sidekick invokes klaviyo_flow_performance, returns real data — works perfectly

Chat session B (next chat, same browser tab, refreshed): same prompt, Sidekick responds “the skill isn’t available to me in this session”

When I asked Sidekick to invoke a specific tool ID, its exact words were: “every attempt to activate the Weaver skill has returned ‘Skill not found’”

So the registration appears to be there for some sessions and missing for others. We’ve also tested on a freshly created dev-dash dev store (same org) and saw the same failure — but with the same chat-to-chat intermittence.

Is there a known propagation/cache layer on the Sidekick side that could be in an inconsistent state for our app (c64a35d9e6e89ce367704b9cf1add9e4)? We had a brief window with a malformed deploy (empty inputSchema.properties on one tool) which we suspect may have poisoned the registry during that window, and the fix is now propagating unevenly. Both issues have been fixed in multiple subsequent deploys (current active: weaver-13).

5

Hey Liam, was about to look at using these myself, perhaps the docs should have a banner mentioning that they’re still closed (other pages normally do for things that aren’t publicly available).

6

Not sure about closed beta, is there an application link anywhere for it? Or is it invite only for now?

UPDATE:
So today it seems to be working again for the most part.

Is there any word on the intermittence of this? or is it just behaving this way because I am not part of the closed beta?

7

Hey Liam, is there a way to sign up for the beta? We would love to explore this for one of our apps. :folded_hands:

8

Hi @tom_digismoothie - the team running the beta are not accepting new applicants. However you should be able to test Sidekick App Extensions with apps when they’re installed on dev stores. It’s just not in General Availability currently.