We are currently struggling to understand how the Shopify App Store rating system behaves in practice.
We understand that the displayed rating is not a simple arithmetic average and that Shopify uses a weighting model based on more recent and “high-confidence” signals. However, the actual outcome still feels difficult to reconcile with the visible review pattern.
A recent example from our side:
-
On Sunday, we received two new 5-star reviews.
-
Before Sunday, our app had already been showing an average of 4.8 in the App Store.
-
Today, we then received a single 1-star review caused by a misunderstanding that we are currently resolving directly with the merchant.
-
Immediately after that, the displayed average dropped to 4.7.
What makes this even harder to understand is that between an earlier 1-star review and today, we received a total of four 5-star reviews and no other ratings in between.
From our perspective, this raises a fair question:
How can multiple recent 5-star reviews have so little visible stabilizing effect, while a single 1-star review appears to move the displayed average so quickly?
We fully understand and support the idea behind weighted ratings if the goal is to reflect recent merchant experience more accurately. But in cases like this, the result feels disproportionate and not very transparent from a partner perspective.
We would really appreciate more clarity on questions like:
-
How strongly are recent negative reviews weighted compared to recent positive ones?
-
Are there thresholds, confidence multipliers, or other mechanisms that can make the displayed score react this sharply?
-
Is there any public documentation that helps partners understand why the visible rating may differ so much from the apparent review trend?
We are not questioning the need for a quality-focused rating system. We are simply asking for more transparency, because otherwise it becomes very difficult to understand or explain these rating movements internally and externally.
**EDIT: Another App in the same category looks like this:**
Based on the rating counts shown in the screenshot, the actual average is 4.62, which would normally round to 4.6, not 4.7. That is why we are struggling to understand how the displayed score changes so sharply in response to individual reviews.
3 Likes
Hey
App Store reviews are weighted, as you mentioned. From what I’ve been able to gather, the recency of a review seems to have the biggest impact on how strongly it’s weighted. I’d also expect there are some trust signals involved as well, such as how long the merchant actually used the app before leaving a review. Looking at your app, I can see most users had been using it for quite a while before reviewing, which is great. I’d also imagine signals like whether the store appears active are taken into account.
I doubt Shopify would share much detail about the exact scoring, as that information would be extremely useful for review farms, which are already a significant problem.
I can see the 1-star review is still the most recent on your app, so it will be interesting to see how quickly the overall score shifts back up once a few more positive reviews come in above it. I do sometimes wonder whether 1-star reviews are weighted more heavily than higher ratings.
Personally, I think the review layout on your app looks great. Having a few negative reviews is completely natural and can actually help build trust, as it shows the reviews are genuine rather than a wall of perfect 5-star ratings.
5 Likes
Hi,
thanks a lot for your thoughtful reply and for taking the time to share your perspective.
What you’re saying makes sense, especially regarding recency and potential trust signals such as store activity and how long a merchant has actually used the app before leaving a review. We also understand why Shopify would likely keep the exact weighting logic private.
What surprised us was simply how strong the immediate impact was. We had already built up a solid average before, received additional 5-star reviews, and yet one recent 1-star review caused a very noticeable drop right away. That felt disproportionate from our point of view, especially since the 1-star review appears to be based on a misunderstanding that we are currently clarifying with the merchant.
That said, we really appreciate your balanced view. You’re absolutely right that a completely perfect wall of reviews can sometimes look less authentic, and in general we also believe that a realistic mix of feedback can increase trust.
We’ll keep a close eye on what happens next — for example, if a new 5-star review comes in and how that affects the overall rating — and we’ll share an update here in the forum thread once we have a clearer picture.
Thanks again for your honest and constructive input — much appreciated.
Best,
Stefan
2 Likes
Basically a 1 star review by a mentally unstable user who never sold a product, offsets 100 5 star reviews from real store owners
5 Likes
That’s also your experience so far.
2 Likes
Geez, around less than 24 hours after I posted this, someone on Shopify Plus left a 1 star review with no text. After relentlessly trying to contact them, they finally replied after 3 days and said they left the review “by mistake”
