Destroying trust in App Store reviews - proof enclosed

Eleven days ago this forum announced “Strengthening trust in App Store reviews.” Since then, genuine, hard-won 5-star reviews have been unpublished from our listing (Event Ticketing, Built for Shopify), while every one of our critical reviews remains in place. The stated goal was to make the reviews that remain carry more weight. For us the opposite has happened, and I can prove it.

What we did

The Partner API does not expose reviews, so the only complete record of the reviews we have ever received is Shopify’s own Partner notification emails. We reconstructed every review from those emails and compared them against our current live listing. Then we cross-referenced each unpublished review against our own records and against Shopify’s own usage billing.

What we found

Multiple 5-star reviews unpublished. Zero negative reviews removed. Every unpublished review we could identify came from a real, active, paying merchant, several of whom sold tickets through Shopify today.

Two cases, redacted. Both are genuine reviews from real, paying merchants. Both are gone.

Case 1 (screenshots below), the timeline:

  • Shopify’s Partner notification that this merchant left a detailed 5-star review in May 2026.
  • The public Wayback Machine snapshot of our listing from June 10, 2026, which still shows that review live.
  • Our listing today: it is gone (verified across all pages on 2026-07-17).
  • Shopify’s own Partner billing, showing this merchant billing per-ticket usage charges, with tickets sold today.

A 5-star review, on the public record as recently as 26 days ago, from a merchant Shopify’s own systems show is active and selling right now, unpublished with no notice.

Case 2 (screenshot below), the stakes:

A merchant who has used our app for six years. Shopify’s own Partner dashboard shows this single store has generated $36,034.40 in app earnings. They are still installed, on a Grow plan, and sold tickets through Shopify this morning. Their 5-star review is gone.

I have redacted both merchants and will hand staff the unredacted specifics for these and every other case.

For context: the public web archive shows our listing at 38 reviews on June 10. Through June we earned four more genuine reviews, all in our Partner notification emails, which put us around 42 by early July. Today we are at 34. The listing was not stalling. It was growing on real reviews, and then it was cut by about eight, including one of those four June reviews.

On the two stated mechanisms

On incentivization: we have never offered, unlocked, or withheld any app feature in exchange for a review, by the definition in the announcement. If an automated classifier flagged us, it is wrong, and we have no way to know or contest it, because there was no notification.

On untrusted-review detection: the announcement says “some genuine apps may see reviews unpublished too” and that “this isn’t a reflection of wrongdoing on your part.” That describes exactly what is happening to us, and it concedes that these reviews are genuine. So genuine reviews from real merchants are being destroyed, by Shopify’s own admission, with no notification, no appeal, and no way to get them back.

Why this destroys trust instead of strengthening it

We are not alone. The thread announcing these changes is full of the same story: developers reporting 30 legitimate five-star reviews gone in an hour, reviews from merchants who used their app for five years, reviews lost in the middle of a Built for Shopify review. There is still no staff response, no appeal path, and no notification.

The announcement said the goal is for the reviews that remain to carry more weight. Look at what is actually surviving. Across the store, five-star reviews from merchants who used an app for two or three minutes, with no real content, are sticking, while detailed reviews from merchants of five and six years who are still actively selling are the ones being removed. Shopify’s own guidance says it prioritizes reviews from established merchants with meaningful detail. This is doing the opposite.

For us specifically, the reviews that survived are disproportionately the negative ones. A listing that keeps the criticism and the two-minute “looks good” reviews, and deletes the verified, detailed praise from long-term merchants, is not more trustworthy. It is less accurate.

This morning a Shopify staffer posted on X that we can appeal by sending support a list of the reviews we want restored. I appreciate the response. But it has the gap everyone is already pointing at: you are asking us to submit a list of the reviews you removed, without ever telling us which reviews you removed. I only know mine because I spent today reconstructing them from your own notification emai
ls and a public web archive. Most developers cannot do that. An appeal that depends on the developer already knowing what was taken is a puzzle, not a process.

What we are asking for

  1. Notification. Tell each developer exactly which reviews were removed, and under which mechanism. The appeal announced on X is hollow without it. You cannot appeal a list you were never given.
  2. A durable, official process with real replies, not a tweet. Developers in these threads report appeals that fell on deaf ears with no response.
  3. Reinstatement of genuine reviews from merchants who are verifiably real and active. My list is ready, with the shop domains, dates, and your own billing records to verify every one.

Every one of these reviews was earned one merchant at a time. So the real question isn’t just for Shopify, it’s for the rest of us: is this the game we accept?

Jeff
Event Ticketing (Built for Shopify)

“All in the game” - Omar

I completely agree with this. We experienced the same thing and lost four genuine 5-star reviews from real customers. Those reviews weren’t easy to earn. We spent weeks, and in some cases months, working closely with those merchants to build trust and deliver a great experience.

Shopify needs to create a transparent appeal process for app developers. Removing legitimate reviews without notifying us or giving us a way to appeal is unfair and incredibly demotivating. We invest significant time building relationships with our customers, and seeing that hard-earned social proof disappear overnight feels like all that effort was thrown away.

I understand this is a new process, but it’s clear there are legitimate cases where genuine reviews are being removed. Shopify should improve the process by notifying developers which reviews were unpublished, explaining why, and providing a fair path to have valid reviews reinstated.

How to find out which of your reviews were removed (since Shopify won’t tell you)

  1. Your only complete record is the Partner review-notification emails from noreply@shopify.com. Search your inbox (two subject formats over the years: “New N-star review for [app] by [store]” and “[app] Review by [store] published”). Label them and export via Google Takeout as mbox. That’s every review you ever received.
  2. Scrape your current live listing (apps.shopify.com/[your-app]/reviews, every page). That’s what survived.
  3. Plug it into your friendly AI agent. The difference is your removed set.
  4. For proof of when something was live, the Wayback Machine (web.archive.org) has past snapshots of your listing.
  5. cross reference your own records for still active merchants - those are your strongest candidates for appeal

This actually means that some apps who did nothing wrong will see reviews removed because the merchants who left them are not “trustworthy.”

that makes sense. If a fraud store leaves reviews for money, they likely also added reviews to random other apps to make themselves look more legit. They didn’t get paid for every review.

but where Shopify messed up is that they removed reviews from merchants that are trustworthy.

for example: we lost 60 reviews and 48% were from Shopify plus stores. There is no scenario where you can really argue that mega brands are engaging in review farming. Period.

What Shopify is basically saying is that these plus stores are untrustworthy.

Agree with the sentiment here, I’m quite new to this game but so far every 5 star review I’ve got has been a genuine battle. You need to bend over backwards for a merchant to get them, and then they still only give them a small percentage of the time. Losing any of them would be devastating.

I think there needs to be a process to reinstate these if evidence of the merchant interaction leading up to the review can be provided. For example, I can provide long emails chains backing up every 5 star review I’ve ever gotten.

Hi folks,

In a rollout today to improve the reliability of reviews we made a mistake and some merchant reviews were incorrectly unpublished. We’re working on correcting this now, more details here:

https://x.com/jzazove/status/2078284775459000349