Metafield values limited to 16KB

Changelog
21

Hey folks!

Thanks for all the feedback and dialogue here. In hindsight, we should have shared our intent and reasoning earlier, before the changelog post. We’ll take this back to the team to ensure the community has more opportunities for input before we announce consequential changes like this.

As for the change itself: as James mentioned above, we’ve been facing significant challenges with large metafields impacting storefront performance, and we need a solution that mitigates these issues.

Limiting metafields to 16KB is the simplest safeguard. We considered alternatives, but they would make the metafields/metaobjects model more complex and harder to reason about.

Next steps: We want to better understand your use cases and discuss options. We’re open to revisiting this decision and finding an approach that strikes the right balance for the community and Shopify. We’ll be emailing all impacted apps to schedule time with us, but you can also book directly here: https://calendar.app.google/BksRAb5nUcgSoGyN8

To be clear, we want you to rely on metafields and metaobjects for data storage. That has been driving a lot of our recent investments, like providing a larger and more predictable number of entries for app developers. It would be counterproductive if this decision made you reconsider, so we’re incentivized to find a path forward that works for everyone.

11 Likes
Reduced metafield value sizes in Shopify
22

Just to start the conversation. Why are people using a meta field as a database vs using metaobject definitions as a table definition and metaobjects as rows?

23

Because I built my apps before metaobjects and migration is a pain / not simple.

3 Likes
24

Metaobjects are just a group of metafields, so the 16 KB limit applies there as well. People use them instead of their database or S3 storage because of performance, Liquid access, fewer request calls, and reliability under heavy load. I don’t want Cloudflare to go down along with my R2 bucket on an important sales day while Shopify is still up, and then have someone who used my app upset because it failed them on such an important day.

3 Likes
25

Metaobjects are just a group of metafields, so the 16 KB limit applies there as well.

@webx47 This might already be clear, but wanted to clarify if it isn’t: the 16KB limit would only apply to individual metafields or individual fields on a metaobject. You would still be able to have multiple fields—or fields in a metaobject—that collectively exceed 16KB.

26

especially for JSON metafields, there are scenarios where migrating to metaobjects just doesn’t make sense. we’re saving product configuration configs (think of it as custom options on steroids) within metafields. each of those is inherently different in structure, making mapping them to a metaobject definition nonsensical.

I get where you are coming from - but this feels like you’re throwing functionality that existed for years under the bus, without a clear indication of what this actually is trying to improve, beyond broad statements

8 Likes
27

@Darius-Shopify I meant on an individual level.

I also agree with @Kevin_Sieger points; let’s say instead of a single big JSON, I create a metaobject to use as a reference. It’s going to be a pain in the future when I need to change the shape of my data because, right now, to migrate I just need to modify and update individual JSON field. but on this “reference route,” it’s going to be such a big pain when dealing with migrations.

2 Likes
28

Even using MetaObjects, wouldn’t the same amount of data be dumped in the storefront?

I believe that for this exact purpose it would be better to have more liquid tooling for handling objects.

29

Metafield values being limited to 16KB is way too low especially for a JSON field. Please reconsider the decision.

Reducing out of the blue from 2MB to 16KB (125 times less!) is another very strange decision, which definitely puts partners in a difficult situation.

4 Likes
30

Thanks for all the feedback folks. There has been some good feedback re simplicity / flexibility of working with JSON that we’re taking into consideration.

We’re discussing options. Will share more details early next week

7 Likes
31

Hi,

I have an app that enables merchants to track unique details about individual items, such as serial numbers, warranty durations, and any other business-specific custom information: IMEI, lot numbers, supplier SKUs, you name it. Some merchants track multiple serial numbers per item, i.e. one serial per part.

All items (sold or on hand) are stored in the app’s database, which is the source of truth. It enables merchants to filter items by their unique references in the app.

However, this data can’t be accessed in other parts of Shopify, e.g. notifications, Order Printer app, etc. Merchants want to share these details with their customers for support and warranty purposes. So, the app stores an array of sold items in Shopify order metafield. Additionally, parts of the app access this data through metafields to improve performance. I love how quick the Shopify admin API is, and I’ve been thinking about using metafields for this purpose even more.

I’ll admit that four years ago, when I released the app, I wasn’t sure how the app would evolve, if at all. So, I kept the metafield schema simple, but verbose - a 2M character limit felt safe. Items are stored as an array of objects with the following properties:

  • Item ID (as stored in the app’s database)
  • Serial number (built-in field)
  • Warranty duration
  • Merchant-defined custom fields: an array of key-value pairs
  • Shopify’s numeric line item ID (no metafield for LineItem)
  • Shopify’s numeric product variant ID

For brevity, let’s assume an item currently takes 160 bytes to store (not unreasonable, and may be larger). That’s 100 individual (not line!) items per order, which is probably enough for most merchants most of the time, but laughable for larger B2B businesses, who are showing more interest in the app as I keep improving it and adding new features.

Sure, I could normalise the data and split it across multiple metafields, maybe use metaobjects for some custom fields, but even then, it may not be enough given the 16KB limit.

Let’s assume an average serial number is 16 characters long. That’s ~1000 serial numbers per order. I started thinking about changing the schema last year, when one client hit the 2M-character limit with 50,000 items, or a similar number, multiple times. The client uninstalled the app, but there are still a few orders with over 1,000 individual items. I’m not only worried about the code changes, additional complexity, and data migration, but also about all the clients impacted by the app’s schema changes. Surely, the competitor apps will be affected by this, as well.

For a solo developer like me, it’s becoming really difficult to keep up with the breaking changes these days. First, forced migration to the web components lib + Preact for extensions. Now this…

Please, Shopify, reconsider this!

Thank you

4 Likes
32

Hi Darius,

I completely understand the negative impact that metafields can have on store fronts. There are also metafields that are not impacting the storefronts, it would be pretty useful to not limit those, for example the order level metafields.