App Reviews AI Filtering is Broken

How can a shop install an app for 3 minutes and leave a 1-star review without getting flagged by the new and shiny AI filtering you guys have.

I see all types of reviews getting filtered out (often 5 star reviews!), but of course the AI is all for leaving up fraudulent 1 star reviews.

Go to their domain: https://agnesestudios.com/

It’s not even an e-commerce website - maybe an agency? Maybe a competitor? It’ password protected no content - probably never will have content!

There’s so much more you guys can do to improve the quality of reviews.

  1. If store is password protected and/or never made a sale and/or brand new store and/or new merchant - block the review

  2. Did they have the app installed for less than X minutes? Block it

  3. Did they actually even use the app? (Check graphql / rest api usage maybe)? Block it.

  4. Does the merchant have a tendency of negative reviews (probably a scammer or a hater) - block it

  5. Does the 1-star leave any text explanation to back up the claim? Block it

In the very least scrutinize 1-star reviews as hard as you scrutinize 5-star reviews. I know you guys are alway triple checking 5 star reviews - please do the same with 1-stars!

I have lots of examples of reviews from questionable merchants getting thru - we put in a ton of work into these apps and we know most merchants have review fatigue - but there’s always some wacky merchant ready to leave a 1-star - for whatever reason - just block them - especially 1-stars because they are mostly ridiculous.

With a 2 or 3 star - you know at least the merchant put some thought and reasoning into it!

3 Likes