I am developing my first app that is embeded in Shopify admin. I am using all the best practices reccomended by Shopify, React, Polars, NodeJS. My app will be optimizing and automating products when it comes to titles, description, SEO description and alt text on images, depending on what user picks in automation.
The thing is when user runs the first product scan, I import all of his products , analyze them, give them seo and fullfilment score. Then if they want they can pick which one to optimize. Right now everything works as intended, but if user has over 1000 products this can turn into a issue since I dont have long running job handling yet. I was implementing these kind of things with Django using celery and redis and it worked pretty good. What are the best alternatives for this Shopify stack? Curious to hear your approach.
You didn’t mention your current hosting, but I’ll assume it’s Vercel, AWS Lambda, Google Functions or Cloudflare workers.
All of these have an execution limit of some kind. I too prefer a serverless environment for the real time user interactions. But the downside of unlimited scale is the lack of long running processes.
You can handle it in many different ways.
Set up a pagination style API route that uses a database to mark progress, then you just continuously call it on a schedule
Self host with something like BullMQ + Redis
Use a managed service like Temporal (you still have to self host the execution layer)
I’ve tried all approaches, I’m most happy with Temporal, but it is expensive if you’re just starting out. But it does have a 100% self hosted option I believe.
Yeah right now for this app I got no hosting yet, on my previous Django app I use railway that manages all the hostings for me, there I got services like Redis and standalone celery that runs.
I am considering the BullMQ and Redis. Any difficulties you had with those?
Unfortunately, Gadget.dev is shutting down March 31, 2027, with 100,000+ store needing new hosting.
The good news: Teifi Bridge already runs in production behind most Teifi B2B and retail builds, connecting Shopify to systems like NetSuite, SAP, Microsoft Dynamics 365, Acumatica, Epicor and P21. It takes the place of your Gadget app in its own isolated runtime, with:
• Every Shopify API change handled by our team as it lands
• 99.5% uptime, written into your contract
• One fixed monthly price, however much you sell
• Your store, ERP, WMS and 3PL left exactly as they are
For the 1000+ products scan I’d skip paginating through normal queries. Admin GraphQL bulk operations (bulkOperationRunQuery) run the export on Shopify’s side, and you read the JSONL result once the bulk_operations/finish webhook fires. Then push the title and alt text updates through a queue like BullMQ so you can control the rate.