Why you should not rely on reviews only to improve your app

While reviews can provide you with valuable insights for improving your application, relying solely on them will leave you with some serious blind spots. Here is why:

1. Extreme opinions are often over-represented

People are most likely to write a review when they are extremely happy or extremely unhappy ("extremity bias"). Also, often only a tiny fraction of your users will leave a review. Rates like 2% or less do not seem to be uncommon*.

In other words: app reviews do not tell you what the silent majority (with the moderate views & experiences) think of your app.

2. App reviews might not come from your "average user"

To make things worse, that 2%* leaving a review is likely to be above average internet savvy and above average involved with your brand.

Especially when new features are requested, you need to be careful as some can come from power-users with very specific needs. You don't know how many (or how few) of your users actually need this feature.

3. You will miss out on a lot of opportunities to improve your app

The vast majority of usability problems do not make it into a review because:

A) People have better things to do than write a detailed report of all the ways your app could be improved

B) Users are not consciously aware of everything they do while using your app.

People use your app to accomplish certain goals, not for self-analysis. Observation and analysis is the job of user researchers.

For example, there might be a screen in your app on which users systematically search around the page before clicking on the 'right' button. While this decreases the quality of their overall experience, they are not always consciously aware of it, so they cannot report it. However, if you were to observe them using your app, you would identify this issue.

C) Users find workarounds

Surprising as it may sound, users often find solutions to your application's shortcomings. Since they manage to reach their goals they will not write to 'complain' about it. Again, if you'd observe them using your app you would realize that you can actually make things so much easier for them.

D) People think it's their own fault

When users do not manage to use an app successfully, they will often assume that it is their own fault. They may get the impression that they are not tech savvy enough and feel embarrassed about it. They are therefore not going to write a review about it.

And last but not least:

4. You will lack the kind of insights that lead to innovation and customer delight

If your only feedback source are reviews, you are placing your app’s strategy at the mercy of what a tiny fraction of your customers decide to actively share with you. Your stance is thus reactive instead of proactive.

Innovation rarely comes from simply reacting to customer feedback. It often comes from discovering the hidden and unaddressed needs; those needs that your customers are not yet aware of and cannot write about. It lies in the solutions that your customers cannot imagine and thus cannot request. That is why research-tools such as in-context observation and interviews can be so powerful. That is why UX designers are needed to imagine these solutions.

Conclusion

If you aim for excellence and customer delight you cannot solely rely on your application reviews. You need to put your app reviews into perspective using analytics and UX research.

NOTE

* I could unfortunately not find any benchmarks on this. 2.5% is the highest proportion of reviews per download that I found among the apps installed on my phone, most were around 1% while 0.3% is one of the lowest I found, without the app being completely obscure (100k downloads). Now of course a) the number of downloads does not equal the number of active users, but that is data I don't have and b) my sample is surely not representative - so please check your own data and make your benchmarks.