App owned Metaobject definitions in the TOML seem flakey

Custom Data
1

Hi team :waving_hand:

I’ve created an app owned metaobject definition in the TOML file which has been deployed to my development store and extension. When I query the metaobject definitions via GQL I can see it present in the results. But when I try to use it, for example by running a metaobjectUpsert mutation I get the response -

No metaobject definition exists for type "app--my_app_id--my_metaobject"

This was working yesterday but it seems that upon stopping the dev server and starting it again today something has been messed here, any advice would be much appreciated.

2

Hey Simon,

The “worked yesterday, broke after restart” part is telling. It seems your caller function changed underneath you.

Try passing $app:my_metaobjectas the type instead of app--<id>--my_metaobject.

The $app: form resolves against whichever app is authenticated on the request, so it can’t go stale. The expanded form is a snapshot of an app id, and the moment that id changes you get what you’re seeing: definition sitting there in query results, but upsert says “no can do partner.”

You can check the mutation, just run this next to the failing upsert:

{
  currentAppInstallation { app { id } }
  metaobjectDefinitionByType(type: "$app:my_metaobject") { id type }
}

If app { id } doesn’t match the number in the app--...-- type on the definition you’re seeing, then you’re calling as an app without ownership. That’s fine for reading, but writing, not so much. That’s why the query looks healthy and the upsert doesn’t.

If none of this is helping, let us know your setup, but a common use is the GraphiQL app in the
admin, where $app: resolves to GraphiQL, not to you, so it fails there no matter how correct
your app is.

If you’re wondering whether a dev server restart can change which app you’re upserting to, it can. I got burned by this when my ‘shopify app dev’ rewrote my prod ‘application_url’ in the partner dash, leaving it a mess though my repo hadn’t changed. Since partner dashbaord state is mutable, local tooling can change it without any alert, and the TOML in your repo suddenly stops being source of truth.

While in the TOML check that you have write_metaobjects and not
only write_metaobject_definitions.

If all of that didn’t help, again just post back, happy to help with any new issues related to this. Some other threads ran into orphaned entries from a definition that got deleted and recreated, and only Shopify can see that side. They will ask you for the X-Request-ID off the failing response if that’s the case.

Happy bug hunting!

3

Hi @cunningorb thanks heaps for your help here.

I should have provided some of the code I’m using, I am using the $app prefix everywhere instead of the app ID directly, that message is just what is returned when I attempt the upsert mutation, so after the $app prefix has been resolved I think.

I just checked, I do have both write_metaobject_definitions and write_metaobjects.

It’s weird I had the metaobject definition and an instance of it created for testing yesterday, it was all working, I don’t think I deployed anything in between, or switched branch so I can’t really explain why it’s all blown away today either. I’ve used GQL to define the metaobject definitions previously with none of these problems, I just wanted to use the TOML definition so I didn’t have to do a migration on all existing stores.

4

My definition in the TOML looks something like this:

[metaobjects.app.app_settings]
name = "App Settings"
description = "App store specific settings"
access.storefront = "public_read"
access.admin = "merchant_read"
capabilities.publishable = true

[metaobjects.app.app_settings.fields.storefront]
name = "Storefront"
type = "json"

And my upsert looks like this:

mutation UpsertMetaobject($handle: MetaobjectHandleInput!, $metaobject: MetaobjectUpsertInput!) {
  metaobjectUpsert(handle: $handle, metaobject: $metaobject) {
    metaobject {
      id
      values
    }
    userErrors {
      field
      message
    }
  }
}

Where $handle is

{
    "type": "$app:app_settings",
    "handle": "app_settings"
}
5

@simon_b Great detail! This narrows it down.

There’s a known issue where the definition gets deleted and recreated while the entries
don’t go with it, so they survive pointing at a definition id that no longer exists. The
upsert finds that orphan by handle and dies on the stale reference. Shopify’s own
explanation of it is here.
That one surfaces as an internal error rather than your message, so I can’t say for sure
it’s the same bug, but your story fits it exactly: it worked, an entry
exists, nothing in the repo changed, and a dev restart resynced the definition in between.

Two minute test. Run the same upsert with a handle you have never used, app_settings_v2 or
anything. If that goes through, the definition is fine and it’s the old entry that’s
poisoned.

Worth running this too, in the same authenticated context as the failing upsert and right
before it:

metaobjectDefinitionByType(type: "$app:app_settings") { id type }

Null there while your list query still shows the type means the resolution isn’t landing
where you think it is. An id back means the definition is healthy and it really is the
entry underneath it.

If the fresh handle works, the workaround people are using is a per-install suffix on the
handle until Shopify ships a fix. Orphans already sitting on the store can only be cleared
by Shopify support.

Post back either way, happy to keep digging.

6

@simon_b Great detail!

There’s a known issue where the definition gets deleted and recreated while the entries
don’t go with it, so they survive pointing at a definition id that no longer exists. The
upsert finds that orphan by handle and dies on the stale reference. Shopify’s own
explanation of it is here.
That one surfaces as an internal error rather than your message, so I can’t say for sure
it’s the same bug, but your story fits it exactly: it worked, an entry
exists, nothing in the repo changed, and a dev restart resynced the definition in between.

Two minute test. Run the same upsert with a handle you have never used, app_settings_v2 or
anything. If that goes through, the definition is fine and it’s the old entry that’s
poisoned.

Worth running this too, in the same authenticated context as the failing upsert and right
before it:

metaobjectDefinitionByType(type: "$app:app_settings") { id type }

Null there while your list query still shows the type means the resolution isn’t landing
where you think it is. An id back means the definition is healthy and it really is the
entry underneath it.

If the fresh handle works, the workaround people are using is a per-install suffix on the
handle until Shopify ships a fix. Orphans already sitting on the store can only be cleared
by Shopify support.

Post back either way, happy to keep digging. Or if you go shopify support route let us know on here how it went. I have had good experience with them.

7

Hi @simon_b, did you happen to uninstall the app and then reinstall before running that mutation?

8

Hi @Paige-Shopify thanks for having a look at this, it seems to be just some kind of glitch with my development environment, there were no uninstalls in between, I just stopped the dev server when I was done for the day.

Probably safe to ignore this one now, I’ve since deployed the TOML changes to production and all seems good.

9

No worries, thanks for letting me know. Feel free to flag it again in the event it pops up again.

10

Hi @Paige-Shopify I’m getting this again on my dev store, making it hard to test a new feature, I’ve tried reinstalling the app but the issue persists, do you have any other suggestions?

Note - there are no issues in production, the metaobject works as expected there, it’s just in development.

12

Hi @simon_b, I understand you have a ticket open with my team about this issue. I’ll mark this thread as unsolved so you can update it with the solution once it’s resolved, in case others run into the same problem.

13

No worries, thanks @Paige-Shopify!

14

Hi is there any update on this issue? I’m facing something similar in my application

15

Same here, less than 1% of shops gets this error but it’s blocking our migration from metafields to metaobjects.

16

@Yash_Chaudhari and @dostu, in these cases, have you noticed whether the affected shop recently uninstalled and then reinstalled the app?

17

@Paige-Shopify thanks for following up. Since Oct 24, 6 shops have hit `No metaobject definition exists for type “app–2935586817–deal_list”` (and `…–deal_block`) on `metaobjectUpsert`. We use the `$app:` type and the definitions are in our TOML.

- 3 of the 6 had installed our app before. The previous install was removed 40 days to about 6 months before they reinstalled, so these were not quick uninstall/reinstall cycles.

- The other 3 were first installs with no earlier install of our app.

For comparison, about 8% of all our new installs in the same period were reinstalls. So reinstalls are over-represented among affected shops, but they don’t explain every case. I can send the shop domains in a DM if that helps.

18

Thanks for sharing @dostu! Do you also happen to have a couple of request IDs of the failed requests? Ones related to shops that haven’t installed the app before would be the most useful.

19

@Paige-Shopify here are request IDs of failed metaobjectUpsert calls (Nov 2, UTC). Each returned No metaobject definition exists for type "app--2935586817--deal_list" (or --deal_block) in userErrors.

  • 23:44:32 deal_list: e651c772-de41-4a35-ba6c-656b108321ea-1793663072 (shop A)
  • 23:46:03 deal_list: 36f7e85a-a226-4452-82c8-6b2e282edd90-1793663163 (shop A)
  • 23:50:49 deal_block: 71ffa3da-6aa0-4b3c-99b7-5735462aa431-1793663449 (shop B)
  • 23:51:00 deal_list: 909a3001-0b37-4baf-9d6b-d18e4299ecf7-1793663460 (shop C)

Correction to my earlier reply: none of these 3 shops had our app before Oct 25, but each one uninstalled and reinstalled it within a day, and the error started only after the reinstall (times UTC, from the Partner API events feed):

  • Shop A: installed Oct 25 10:53, uninstalled 16:32, reinstalled Oct 26 09:39, first error 09:44
  • Shop B: installed Oct 25 17:39, uninstalled 18:29, reinstalled 18:30 (first error 18:30), uninstalled again 18:53, reinstalled 20:27
  • Shop C: installed Oct 25 12:19, uninstalled 22:16, reinstalled 22:18, first error 22:21

So all of them match your reinstall theory, and the definition is still missing a week later. I can DM the shop domains.

20

Perfect, thanks for sharing those @dostu. We can derive all the information we need using the combination of that timestamp and request ID.
I’ll provide an update here once our investigation on this data is complete.