Apple just provided the perfect example of why you can’t trust App Store review scores

You pissed off people by somewhat breaking your app, and they’re leaving angry reviews. How can you salvage your reputation? Apple just found one incredibly effective way — get listeners to submit better reviews by interrupting their podcast experience with an in-app prompt to submit a rating.

That’s how the Apple Podcasts app went from a publicly embarrassing 1.8-star score all the way to 4.6 stars in a little over a month without any actual fixes, as developer and App Store watchdog Kosta Eleftheriou points out. And it’s still going up: according to AppFigures data, the app has been getting thousands of ratings every day since November 9th, with the vast, overwhelming majority of them issuing a 5-star score.

The app has made it to 4.7 stars overall as of this writing and is firmly the No. 1 App Store search result for “podcast.” It looks far more desirable to a new user than it might have before.

If you think there’s a perfectly reasonable explanation for this, you might be right — it could definitely be that people who bother to submit reviews tend to be angry, and a lot of people who love Apple Podcasts and never bothered to look it up in the App Store (remember, it’s preinstalled!) are finally balancing things out.

But do those people actually love Apple Podcasts? Because if you really look at the reviews, it seems like some funny business is going on. There are new, positive reviews, but they aren’t reviews of the Apple Podcasts app at all — they’re reviews of podcasts themselves.

I kid you not — this is the top review for Apple Podcasts on the App Store:

Screenshot by Sean Hollister / The Verge

Here are a few more of the “Most Recent” reviews of the Apple Podcasts app:

• “Amazing show! Hilarious and well researched,” writes SammyAls, adding, “The dynamic is amazing, and the content is SO needed! Love this.”
• “Mobley has Depth and Insight,” writes xbacksideslider. “Nice to listen to thoughtful and factual podcast. Far from the superficial emotional appeals to envy and self congratulating faux empathy that so dominate popular culture.”
• “The table,” says Jkimble6091. “Being a future young millionaire listening to Anthony Oneal keeps me on track during all the ups and downs of life.”

I wondered if maybe this was a common confusion with podcast apps, where listeners think they’re reviewing a podcast instead of the app itself. But no, I didn’t see that obvious pattern when I checked reviews for other top podcast apps in the App Store. Almost every review on competing apps was a review of the apps themselves.

Apple confirmed to The Verge that it’s using a new prompt but claims it’s nothing out of the ordinary. “With iOS 15.1 released last month, Apple Podcasts began prompting listeners to leave a rating and review just like most third-party apps — using the standard Rating & Review prompt available to all developers,” a spokesperson tells us.

We weren’t able to track down a copy of the prompt ourselves to confirm when and where it appears or what it looks like — which seems important if people are getting confused — but it is indeed a standard feature of the App Store, one you can even turn off if you like under Settings > App Store > In-App Ratings & Reviews.

But intentional or not, standard or not, the problem with star scores is there’s no way to tell whether they’re legitimate. We don’t know if someone pressed a five-star button because they loved the app, or thought they were rating the podcast itself, or just wanted to close the prompt as quickly as possible. We don’t know if Apple is prompting everyone, or just its most dedicated fans, or some other algorithmic subset that just happened to give it an advantage. Some bad actors reportedly even buy star scores for their egregious App Store scams, and it’s impossible for most App Store shoppers to tell. We’ve even seen an iOS app that refuses to open unless you give it a good score.

These are the reasons why I suggested that Apple should lead the industry by killing off star ratings for good, among other things it could do to prove it puts people ahead of profits in the App Store.

But with Apple Podcasts, the company is using the same broken star score system that uplifts scammers for its own benefit as well. And it’s a crystal clear example of why you can’t trust star scores — because everyone knows this was a 1.8-star app just last month with many valid lingering complaints, and nothing’s fundamentally changed. It’s the exact same app today as it was then.