I’m going to be super frank:
TLDR: Shopify in all fairness you botched this up. You had good intentions - I believe, but you really failed and should take responsibility.
First, the good:
- Shopify listened and recognized they had a problem and had to address it.
- Shopify still seems to be open to fixing it.
The problem: As the app store grows, bad actors are increasing. The old issue of abusers are getting worse, and the time to do something could no longer be pushed off.
As I see it you have two types of bad actors in relation to reviews:
- Review Farming: Basically fake reviews from stores that may seem real, might be real etc.
- Incentivized Reviews: Requiring reviews to unlock features (ie. remove branding), broken features etc.
Shopify had to solve this, and so they seem to have spent time analyzing and purging reviews. We lost 60+ reviews (17%). 48% of those were Shopify Plus merchants. Merchants who are well known established brands, where there is zero chance they are engaging in review abuse. We can barely get them to take the time to leave a review altogether even when they love us.
What this told me is that whatever method shopify deployed was flat out broken.
My assumption is that they tried to deploy some kind of review velocity method, that would flag reviews that arrived together in a period of time. But that is wrong…
The policy does not demand that apps can’t request reviews… we can. We can not incentivize them, or request specifically a 5 star review. So for example, Navidium (our app) we didn’t used to ask for reviews daily.. 2-3x a year we would reach out to our key merchants and ask them if they want to leave a review. The result would be us getting a flood of reviews in a few days from super loyal happy merchants rather than a steady stream.
Not against policy in any way… but perhaps a bad method strategically.
___
**My method:
**
I always suspected a competitor app of engaging in review farming. Now I am positive of it.
The app is Captain Shipping Protection (out of China) and during this purge they actually increased reviews! So I had Claude work with me to crawl their reviews and ours. I also ran route protection because they are the closest competitor to the two of us.
Now why is this so powerful: Both apps exist in the same space. We both operate basically the same business.
Merchants who install, in most cases need expert help to get the shipping protection widget set up properly on their store. Often it requires dev resources (we provide it free) to install the widget in 3rd party carts etc. It takes time, sometimes a few hours to a few days. You have to request partner access, get access, dupe themes, install, send back to merchant for review, etc.
Even if you have auto install (we do) it doesn’t always work with every theme, 3rd party cart, etc. It also takes time to get fonts, colors, styling, price and fees done. Basically it takes time.
Navidium: 1 review left within 1 hour of install (0.3%)
Route: 1 Review left within 1 hour of install (0.3%)
Captain Insurance: 81 reviews left within 1 hour of install (32%)
*Same app - same business model. How does that make sense? It doesn’t.
How about reviews within 1 day:* Sort of reasonable, eh?
Navidium: 4 Reviews (1%)
Route: 1 Review (0.3%)
Captain Insurance: 135 Reviews (54%)
So again: Stark difference here: Because frankly like I said earlier, it usually takes time to do things right. People don’t just drop a review immediately the same day. They don’t even know if it’s going to work for them properly… They need orders to come in to see if they come in “protected” etc etc. It’s why we barely get any reviews that fast, but somehow they get 54% the same day as install? *
___
Another example: Rating only reviews
*
Granted we don’t really have control over this, and I’m not even sure why Shopify allows it to begin with…
But:
Navidium: 11 Reviews (4%)
Route: 5 Reviews (2 %)
Captain Insurance: 72 Reviews (29%)
So 29% of all their reviews didn’t even bother writing anything and just leaving 5 stars with zero actual content.
_____
What I am getting at here is a clear methodology. There are categories where leaving a review within minutes, hours, or even days = clear abuse. When you run a few apps across each other you shouldn’t see such stark differences, and if you do, well… now you have a pattern you can follow.
This method would actually begin to start finding bad actors. Actual analysis not wide fast hacks…
Every category has it’s format, but you can easily find the outliers… you just have to be willing to look.
