User Research: Improve user retention

I've worked for a data-driven start-up who provides an app with entertaining content and in-app purchases. The qualitative feedback they got from their users was very positive, people loved the app and spend money on a regular basis to get more access to the entertaining content in the app.

After analyzing the app usage of their users, the data team noticed that users stop using the app after only a short amount of time and never come back. Some of the users churned without spending they coins they have purchased before to buy more entertaining content on that app.

What was going on?

As this start-up operates data driven, there were plenty of data available. The issue with data analytics is: Data can show you there is a problem, but not what the problem is. If you want to fix the problem, you need to know what the problem is and why it is a problem.

I was hired as a user researcher to find out why people churn and how to improve the user retention.

According to the Double Diamond model, I focused on discovery and exploration of the problem space, defined priorities and started to recommend solutions based on the insights.

My focus is on qualitative methods to truly understand what users do and why they do it. It is important to talk directly with users to understand their pain points in context and also to check for contradictions in what they say as people are usually not fully aware of what reasons led to what decision. User interviews are essential for understanding people's behavior and are my main source for insights. Additionally, I conducted surveys to collect a high number of answers to be able to quantify the findings. To minimize bias by given answers, I first collected quotes of churned users using surveys with open questions. After deriving reasons why users decided to stop using the app from interviews, I conducted quantitative surveys to add proof to my insights from a high number of users.

First I took a look at data that was available already. They had done some user surveys before as well as some short interviews which turned out to be a good source to get a first understanding of what might be going on. This helps to not start digging into the dark, but to start with some interesting points that could be further explored. I also talked with customer service about problems users face, read reviews of the app and used the app myself. To get a better picture of what aspects are most important for the business, I collected the views on the problem space from different stakeholders.

I focused first on interviews with users who already churned to get most relevant information for business development fast. To set the prioritiesright, I started speaking with a user segment with a major impact on business and revenue. I defined the most important segment by high spendings and a high engagement with the app before they suddenly stopped using the app.

Over all 4 segments, I conducted 40 interviews: 30 interviews with churned users, another 10 interviews with users that still use the app daily. I collected a lot of nuanced reasons that helped to narrow down the main reasons for leaving the app and also provides insights for other stakeholders.

While conducting interviews, I sent out the first round of surveys to users of the 4 segments to collect a high number of qualitative data. Instead of multiple choice answers, users had to answer questions in their own words. By doing so any bias by given answers is reduced to a minimum. After some interviews and analyzing the qualitative survey, I was able to send out another survey with multiple choice surveys to quantify the reasons I found why users churn. Together with the qualitative insights, this gave me a good picture on what problems users were facing (qualitative) and how important these problems were (quantitative).

By the end of the project I collected a set of nuanced reasons why users stop using the app in context. Context is important to find solutions that solve the right problem. After talking to 40 people, I was also able to report on consumer psychology, motivation to use the app and the decision making process for choosing this app over others and for how they decide to spend money in the app. For each of the user segment I assembled findings and quotes into personas.