Developer Community Suggestions for Tackling Fake App Reviews

Shopify App Store
1

Ollie here once again rambling on about fake reviews. Surely I’ll get somewhere one day with this :sweat_smile:

A few of us developers, including @abueler, @PaulNewton, @David_Arbias and others, have been discussing some ideas that could potentially help tackle fake reviews, while still avoiding systems that could be weaponised against honest developers.

Here are a bunch of ideas that have come up during discussions. @KyleG-Shopify, @Paige-Shopify, @Donal-Shopify, @Liam-Shopify, @Irene-Shopify, @jzaz, @Wes-Dev-Shopify, sorry, I’ve sort of selected some of these people at random) would you mind reviewing these and passing them onto the review taskforce?

I’m sure many of these ideas are already being considered internally, but I thought it would still be useful to share them from the developer side, as these concerns continue to come up regularly within the community. Hopefully some of the suggestions are useful, or at the very least help contribute towards the wider discussion around improving trust and transparency within the App Store ecosystem.

  1. Closed store reviews should probably be automatically archived again (@abueler )
    One of the biggest concerns raised was that reviews from closed stores don’t appear to be getting archived automatically anymore (can anyone confirm this?). Historically, this acted as a pretty effective self-correcting mechanism. If fake stores can subscribe once, leave reviews, close down, and still keep the reviews active permanently, it massively lowers the cost and risk of review manipulation.

  2. Add trust indicators to reviews (@abueler )
    Shopify could introduce visible trust signals on reviews. For example, showing whether the store has remained active, has genuine order history, or has been subscribed for multiple billing cycles. This gives merchants more transparency, while reducing the risk of false positives against genuine developers.

  3. Weight reviews based on long-term merchant activity (@abueler )
    Reviews from merchants still actively using the app months later should probably carry more weight than reviews from stores that disappear shortly after posting. Genuine merchants tend to stick around, whereas fake review stores often churn very quickly.

  4. Include review integrity checks as part of the Built for Shopify review process (@abueler )
    A lot of developers are concerned about apps with extremely questionable review patterns still carrying the BFS badge, despite BFS being intended as a trust signal for merchants. Including review hygiene checks during annual BFS reviews could help address this without requiring overly aggressive automated moderation.

  5. Bring back visible store links on reviews
    Personally, I’d also love to see visible store links return on reviews. It used to make it much easier to identify whether a reviewing store actually looked genuine. Even now, it’s surprisingly common to see reviews on certain app categories from stores that clearly never realistically used the app at all.

  6. Detect and manually review abnormal review velocity spikes
    If an app suddenly gains a very large number of reviews over a short period without matching install growth, that should probably trigger a manual review. Genuine growth patterns are usually far more gradual and consistent.

  7. Add category relevance checks to help flag suspicious reviews
    Some fake reviews become fairly obvious when you compare the review to the actual store. For example, a blogging app review from a store with no blog, or a product review app review from a store with almost no products. Signals like this could help identify suspicious activity for human review.

  8. Introduce lower weighting for reviews from extremely new stores
    Rather than preventing new merchants from leaving reviews entirely, reviews from very new stores could initially carry less weight until the store has existed for a reasonable amount of time or shown genuine activity.

  9. Add transparency around archived or removed reviews
    One thing that would really help rebuild trust is greater visibility around enforcement. For example, app listings could show when reviews have been archived or removed for policy violations. Right now, it often feels like enforcement either doesn’t happen, or happens silently with no visible outcome for merchants or developers.

  10. Introduce stronger escalating penalties for repeat offenders
    At the moment, the risk/reward balance often seems to favour fake reviews. Repeat offenders could face progressively stronger consequences, such as BFS badge removal, ranking suppression, temporary App Store visibility reductions, or eventual removal for repeated abuse.

  11. Separate support experience from product quality in reviews
    A lot of manipulated reviews tend to be extremely generic and support-focused. Breaking reviews into separate categories like support, ease of use, reliability, and results could make review manipulation much harder to scale convincingly, while also giving merchants more useful information overall.

It’s been mentioned frequently that the App Store should allow the best apps to win. Until the fake review problem is properly addressed, that simply won’t happen. The best apps will never consistently win if lower quality apps can artificially inflate themselves with hundreds of fake reviews, regardless of what other improvements are made to the ecosystem.

Also, if any other developers have additional ideas, observations, or suggestions around review integrity, feel free to add them below. The goal here isn’t to target or call out specific apps, but to have a constructive discussion around ways the ecosystem and trust signals for merchants could be improved overall.

2

Posting some ideas from my previous post here to keep things in one place.

  • Public violation history: Every app page should have a public compliance log. It should list the type of violation (like fake reviews or keyword stuffing), the date, the current status, and a short note from the Shopify team.

  • Algorithmic penalties: Violations need to hit a developer’s bottom line immediately. If an app breaks the rules, Shopify could deploy automatic restrictions like:

    • Pushing them far down the search rankings to kill their organic traffic.
    • Instantly freezing the listing if flagged for review fraud so they cannot accept any new reviews.
    • Blocking their ability to run paid ads in the App Store.

  • Penalty expiration: We need to separate accidental rule-breakers from chronic cheaters. If a developer fixes the issue and stays compliant, their search and advertising restrictions should expire after a few months so they can recover. However, the incident should still stay in their public log, marked as “Resolved,” so the history remains transparent.

  • In-context reporting: Right now, the reporting process is buried in a form that feels built for partners reporting other partners. Shopify should add a simple “Report this App” link directly on the listing page so merchants can also easily flag fraud through a quick popup modal.

3

I want to really stress that focusing on the right incentives is the absolute best way to control this problem.

If we want to change how people behave, we have to change the math of the system. Right now, the financial reward for cheating is incredibly high, while the risk of getting caught is practically zero. It is actually a smart business move for bad actors to break the rules because the payoff is guaranteed.

The system needs to make cheating a massive, immediate risk to a developer’s revenue and their brand reputation. Once the penalties instantly destroy the financial benefit of breaking the rules, it simply won’t be worth the gamble anymore. That is the only way we actually limit this issue in the long run.

4

Hey @Paige-Shopify and the Shopify team,

Could we get some indication that our ideas and feedback are at least being read? Many of us are taking the time to share suggestions because we genuinely want to help improve the Shopify App Store ecosystem, which clearly has some significant issues at the moment.

It’s a little disheartening to put thought and effort into providing feedback, only to receive complete radio silence in return. Even a brief acknowledgement would go a long way and would help reassure developers that their input is being considered.

5

Hi @Ollie_autoBlogger,

First of all, we appreciate the time and thought you and other Partners have put into this.

I’ve been forwarding all your feedback to the relevant teams. At the moment, the only update we can share is in Eytan’s tweet.

6

Great post @Ollie_autoBlogger !! :clap:

I was about to post something about reviews myself when I saw this, excellent.

We’ve been “fighting” with Shopify support for months now, filing reports, reporting partner violations… the reply is always the same: “Some from our Governance team will get back to you”. They never do. I’m starting to doubt if that department/team even exists lol.

I’m not interested in reporting partners anymore; it’s a waste of time. Clearly, there are some apps that have benefits because we have reported them over 10 times (and for different violations), and nothing ever happened.

My concern at the moment is the lack of transparency around archived or removed reviews.
In some cases, when a review is positive, especially a 5-star review, it appears to be archived very quickly once the store is no longer active or no longer meets certain account conditions.

However, when the review is negative, even if the store seems to be inactive, closed, or no longer using Shopify, the review appears to remain visible indefinitely.

We’ve opened tickets about this and have been trying to understand the criteria for more than a month, but we haven’t received a clear explanation yet. So we’re genuinely trying to understand how this works.

Does anyone know what criteria Shopify uses to archive reviews from inactive stores?

Has anyone else noticed that 5-star reviews are removed or archived faster, while 1-star reviews from inactive stores remain visible?

We currently have +3 negative reviews from stores that no longer exist (and filled reviews because they wanted refunds, which we did), and they are never archived.

When 5-star reviews are archived all the time… that’s funny.

I’d love to understand Shopify rules :slight_smile: , and I’d definitely love an appearance from the Governance team, so we know they are actually real people who care about our businesses.

@Paige-Shopify @KyleG-Shopify @Liam-Shopify @Donal-Shopify @Irene-Shopify

7

Hey @Paige-Shopify, thanks for passing on our suggestions!

For any other developers here, please feel free to keep sharing your ideas and feedback.

8

Hey @Ceci_R

It can get pretty disheartening reporting apps. I’m certainly completely over doing it myself these days.

That said, a better way of reporting partners, instead of having to fill in that form that gives no feedback, no meaningful follow-up email, and no real visibility, is on our list of recommendations. Fingers crossed that is one of the changes Shopify makes.

Being able to report individual reviews would also be a great benefit, both for reporting incorrect reviews on your own apps and fake reviews on other apps.

Regarding archived and removed reviews, this is still a very grey area from a developer’s point of view. From what I understand, when a store’s plan is reduced to unpaid, the review is moved into archived, and when it changes back to a paid plan, it is moved back to published.

However, as I mentioned in point #1 of my initial post here (Thanks @abueler ):

One of the biggest concerns raised was that reviews from closed stores don’t appear to be getting archived automatically anymore (can anyone confirm this?).

It seems like there may be some sort of disconnect where stores that close are being missed. Surely these should be treated in the same way as stores that have moved to an unpaid plan?

This one feels important, so @Paige-Shopify can we please get an answer from Shopify on how reviews from closed stores are handled?

For consistent behaviour, and to help reduce fake reviewing, surely stores on non-paid plans and stores that are closed should both have their reviews automatically archived, as is the current behaviour with non-paid plans.

9

I’ve confirmed that multiple factors are considered before a review is archived, and store closure is just one of them. We can’t share further details since that information could be exploited by bad actors.

10

Hey @Paige-Shopify, thanks for confirming that! It would be great if closed stores could be treated in the same way as stores with an unpaid plan. That would seem like a sensible approach and could make it much harder for fake reviews to remain in place.

11

Just to add a couple of points I forgot to mention, which have also been discussed in other threads:

  • A dedicated inbox that we can forward fake review offers to. I personally still receive a bunch of these every day.
  • As mentioned above, the reporting system also needs to be significantly improved. Reporting obvious violations currently feels like such a tax on developers, especially when it often seems like nothing is done. We need simple, one-click reporting. The onus should not be on developers to spend time writing lengthy reports and gathering extensive evidence when many of the violations are so blatant. Shopify should receive the initial report and do the legwork from there.