An update on App Store reviews

Shopify App Store
1

I’m here again to talk about App Store reviews.

Let me start by saying thank you to everyone who took the time to report concerns about App Store reviews through Shopify Support and share your experiences with us. Your reports, examples, and feedback help us shape the trust criteria we use in our review system.

We took time to refine our approach, incorporated feedback from partner appeals, and adjusted the signals we use to assess authenticity. As a result, you may continue to see changes to review visibility on your listings. Some reviews may be restored, while others may be unpublished.

Review quality is something we’ll keep improving over time, and that means the system will continue to evolve. This is part of our ongoing work to ensure reviews reflect genuine merchant experiences and provide a trustworthy signal for merchants.

We’re committed to doing this work with great care, and we’ll keep listening to what you’re seeing on your end.

Thank you for helping us make it better.

2

@jzaz Thank you for listening to the community’s feedback. I have a couple of questions:

  1. How long is the full algorithm rollout/sweep going to take?
  2. If an app gets some reviews restored, does that mean that app is now done? or is the algorithm rollout still going to check its other reviews?

Thanks again.

3

Thanks @jzaz and the rest of the team for this work.

Will Shopify consider introducing new penalties for repeat offenders?

Toy Example: New app in category buys 1000 fake reviews and goes to #1 in rank. Over period of 8 months while fake reviews remain they accumulate large number of installs due to their ranking and secure 200 real 5 star reviews. Another Shopify review purge removes the 1000 fake reviews, but because of the temporary boost they are now net 200, 5 star reviews.

In this example (which has been seen in the wild, maybe not such extreme numbers), the app has won. They have secured a top spot riding the wave securing installs + legitimate reviews. Shopify removing the 1000 fake reviews has been in no way a penalty as they retain a high ranking.

In instances like this would Shopify consider a temporary removal from the public app store? They would retain their individual app install page, but this would never be surfaced through search etc for a period of X months? Many of us believe an approach like this would neutralise the current fake review strategy that far too many are taking.

4

Thanks for putting back some of our hard earned reviews! The disappearing reviews situation was making me want to sell the apps - because unless our pipeline is absolutely focused on gaining/gaming reviews, the count would just go backwards! Soul crushing!

5

Thanks for the update, @jzaz. It reads smoothly, but since this is a developer forum and not LinkedIn, we would really appreciate it if you could stick around in the comments to answer our questions instead of just dropping an announcement.

Could you share some actual specifics on two points:

  1. What concrete steps are being taken beyond mystery algorithm tweaks?
  2. How will Shopify actually enforce the Partner Agreement with real penalties (such as suspensions or bans), given that automated filtering clearly is not solving the root problem?
6

Hey @jzaz, agree with all the sentiment here. Great that we got some reviews back, but overall, fake reviews are still an issue. Me getting like 10 reviews back, while others have brought 100 in the meantime, makes it all feel pretty meaningless. We need harsh penalties, and we need open communication. However, also I know it’s not easy, so thanks for your work so far!

7

Thanks for the update. While it’s good to finally have an official response, I’m quite disappointed with the lack of details. You asked us to ask questions on the original forum topic. Over 60 replies have been posted with questions and concerns, and many remain unanswered. It would great if all those questions would be answered instead of starting a new topic without addressing everything mentioned by many partners.

8

@jzaz Thank you very much for your update. It’s good that we’ve gotten some reviews back. However, as some people here have already pointed out, the actual problem of fake reviews hasn’t been resolved in the slightest. And let’s be honest, we’re essentially back to where we were two months ago. The real problem, the one that’s harming the App Store and that we’ve all been writing about here for months, still remains. :frowning:

9

Hi @jzaz, thanks for the update and for continuing to work on this.

I wanted to ask about a couple of reviews we received, since I’m trying to understand how the trust criteria apply in practice.

Here’s a 1-star rating left after just 4 minutes of using the app:

SCR-20260904-ivys
SCR-20260904-ivys1830×228 10.6 KB

And here’s a 2-star rating left after 6 days:

SCR-20260904-iwgq
SCR-20260904-iwgq1838×214 10.4 KB

Would reviews like these be considered to reflect a “genuine merchant experience,” or is duration/engagement with the app one of the signals factored into authenticity assessment? It’s hard for us to reconcile a few-minute interaction with a lasting quality judgment on the app, and I’d love to understand whether that’s something the new criteria account for, or whether it’s something we should be flagging through Support case by case.

Appreciate the transparency on this - just trying to make sure we understand how to interpret it as partners.

10

Thank you for the message, however:

I’d rather hear apologies for the mistakes you committed.

Just to be clear, we have never asked to be the subject of your experiments with live data that drive our businesses

Legitimate reviews we lost have not been restored. One very specifically from a user using the app for over 1 year. There is no way it would get flagged as not legitimate by a functional algorithm. If you have suspicions it’s not legitimate, get in touch with me and I will be happy to show you why it makes no sense.

Then please communicate better and stop thinking that the silence is a solution.
I’m pretty sure that if you spent more time around actually communicating with us you’d get even more feedback and help avoiding the frustration in the first place.

You made my day @TobiasDalhof :100:

11

One thing that still seems worth addressing is the detection of potentially fake or incentivized reviews.

For example, one app currently featured under “Popular in Germany” has a striking review pattern: many merchants appear to have used the app for less than an hour before leaving a 5-star review, often without any comment.

For a returns/withdrawal app, this seems particularly unusual. It is difficult to meaningfully evaluate such an app within minutes, before actually processing any returns or experiencing the workflow.

The time between installing/starting to use an app and submitting a review therefore seems like a relatively simple “low-hanging fruit” signal Shopify could use when identifying suspicious review patterns.

Another point worth questioning is the “Popular in Germany” selection itself: four out of six featured apps currently appear to serve essentially the same use case. If the goal is to help merchants discover popular apps, a broader mix of app categories would arguably provide much more value.

Example (first on Screnshot): Rezensionen: Erfüllt die EU-Pflicht mit Button, Mail & Dokumentation | Shopify App Store

Screenshot 2026-09-04 at 10.01.10
Screenshot 2026-09-04 at 10.01.101920×1166 465 KB

12

I’ve been watching the EU Withdrawal buttons too. It seems like there was a mad rush of dubious review tactics to get to the top.

Getting 1,200 reviews in a single week and then dropping back to almost none certainly doesn’t look legitimate. Surely activity like that should be picked up by automated checks?

image
image1240×516 37.9 KB

13

Here’s an overview of the questions and concerns raised in the previous thread that I think we all would like to have answered as that thread was specifically opened with “Have questions? Drop them below,” and over the following weeks partners raised a large number of very concrete questions. Most of them still seem unanswered, and unfortunately this new update remains quite high-level.

We would really appreciate some direct answers on the key issues that were raised.

For example:

  1. What are the actual consequences for partners who deliberately or repeatedly manipulate reviews, beyond removing some of the reviews they should never have had? Can apps lose Built for Shopify status, be temporarily delisted, suspended, or have their Partner account terminated, and under what circumstances?
  2. What is the official appeal process when a genuine review is incorrectly unpublished? How does a developer find out exactly which reviews were removed, what evidence can be submitted, will the appeal receive human review, and what sort of turnaround can partners expect?
  3. How does Shopify handle reports of demonstrably fraudulent or malicious reviews? Will developers receive confirmation that their evidence was actually investigated and some indication of the outcome?
  4. What safeguards are being used to prevent legitimate reviews from long-term, paying merchants being caught as false positives? And at the same time, how are obviously suspicious patterns, such as reviews submitted minutes after installation or huge unnatural spikes in review volume, being detected?
  5. Does merchant/app engagement play a role in determining trustworthiness? For example, installation-to-review time, meaningful usage, length of usage, account age, continued usage after the review, or the merchant’s wider review history?
  6. How long is the current backfill/sweep expected to continue, and if reviews have been restored for an app, does that mean Shopify has completed its assessment of that app or can further reviews still be restored or unpublished?
  7. And perhaps most importantly, how does Shopify intend to address the advantage already gained by apps that manipulated the review system? Simply removing fraudulent reviews months later does not undo the additional ranking, visibility, installs and legitimate reviews that may have been gained because of them.

I don’t think anyone expects Shopify to disclose the exact workings of the detection system in a way that would help bad actors circumvent it. But there is a big difference between keeping anti-abuse signals confidential and giving partners no clarity about the policies, enforcement process, appeals process, timelines and consequences.

A lot of people took the time to provide detailed examples and constructive feedback in the original thread because Shopify explicitly asked us to do so. It would be greatly appreciated if the team could go back through those questions and provide substantive answers, even if some of the answer is simply “we haven’t solved this yet.”

Again, the new update is welcome, but without those answers, many of the concerns raised over the last two months remain unresolved.

Thanks.

14

@junaidkbr The work here is never really done as we are going to be looking at new signals and information.

15

I would be happy for my first review - sitting in the no review zone is brutal !

16

I have a good one for you. Already escalated to the support. How do we get it resolved?

We’ve got a special deal with them, so they got a permanent %age discount on the app. Is that why it’s flagged?

17

Selfishly pushing my review signal suggestion. Could help validate the source of reviews, especially if policy requires making those attribution logs always available to Shopify for review at any time. It would definitely help with identifying fake reviews. Reviews with a source (like modals triggered with the Shopify Reviews API or personalized asks in support chats) would be less suspicious than a natural review that had no prior history.

18

@Philo

Q: What are the actual consequences for partners who deliberately or repeatedly manipulate reviews, beyond removing some of the reviews they should never have had? Can apps lose Built for Shopify status, be temporarily delisted, suspended, or have their Partner account terminated, and under what circumstances?

Apps found violating our review policies can face delisting or account termination, not just review removal, and we’re continuing to improve our detection and enforcement standards against these violations. We evaluate each case individually based on the severity and pattern of violations while maintaining consistent enforcement of our policies.

Much of the documentation is public here: https://help.shopify.com/en/partners/help-support/faq/reviews

Q: Does merchant/app engagement play a role in determining trustworthiness? For example, installation-to-review time, meaningful usage, length of usage, account age, continued usage after the review, or the merchant’s wider review history?

We can’t share the specific signals that triggered an action on individual reviews.

Q: How long is the current backfill/sweep expected to continue, and if reviews have been restored for an app, does that mean Shopify has completed its assessment of that app or can further reviews still be restored or unpublished?

There is no end date, reviews are being assessed continuously from when they are submitted and to whenever the information behind them changes. Reviews that are visible today may be hidden over time, or vice versa.

Q: And perhaps most importantly, how does Shopify intend to address the advantage already gained by apps that manipulated the review system? Simply removing fraudulent reviews months later does not undo the additional ranking, visibility, installs and legitimate reviews that may have been gained because of them.

Apps found violating our review policies can face delisting or account termination, not just review removal and we’re continuing to improve our detection and enforcement standards against these violations. Your feedback is a helpful part in that work.

19

Thank you for hard work on this issue. Finally got the review up after an over 1 month hold on it

20

@jzaz thanks for your reply, that’s helpful! Could you answer the remaining questions as well? :slight_smile: I still have reviews that have not been restored and the whole thing is just a black box:

  1. What is the official appeal process when a genuine review is incorrectly unpublished? How does a developer find out exactly which reviews were removed, what evidence can be submitted, will the appeal receive human review, and what sort of turnaround can partners expect?
  2. How does Shopify handle reports of demonstrably fraudulent or malicious reviews? Will developers receive confirmation that their evidence was actually investigated and some indication of the outcome?
  3. What safeguards are being used to prevent legitimate reviews from long-term, paying merchants being caught as false positives? And at the same time, how are obviously suspicious patterns, such as reviews submitted minutes after installation or huge unnatural spikes in review volume, being detected?

You also mentioned the following:

Apps found violating our review policies can face delisting or account termination, not just review removal, and we’re continuing to improve our detection and enforcement standards against these violations. We evaluate each case individually based on the severity and pattern of violations while maintaining consistent enforcement of our policies.

As mentioned by others, we are seeing dozens of partners who have violated the policies yet still have their app listed on the app store. It seems barely anyone is punished?

To give you an example, I reported a partner that used their Shopify store to leave a 1-star review on my app because they launched a competitor app. I have all the evidence to prove this. The 1-star review got removed (not sure if it was Shopify or they did it), but the partner did not got punished at all. For that behavior I would expect delisting for a few months at a minimum. I never got a reply from Shopify on the outcome of my report either, which is frustrating.