Building, testing, and improving features of a Messaging App

Rolling research is a lightweight process for conducting frequent regular research on a common theme. We perform weekly rounds of studies on various aspects of the Message App to determine the adherence and behavior of the Brazilian audience towards possible improvements and new functions to be implemented by the Client.

Gain a comprehensive understanding of how users from diverse social classes and educational backgrounds perceive, utilize, and anticipate the implementation of new features on the Message App. Simultaneously, grasp users' expectations and identify any usability issues that may arise while using a particular feature.

Through research, we provide valuable and actionable insights based on Brazil's context of use to the Client's teams in and outside of Brazil. Understanding user needs and solving problems becomes more achievable by aligning research results with global teams weekly, meaning that the biggest messaging platform used today gets to continue facilitating communication for its users worldwide.

In-depth interviews

Usability tests

A/B tests

Diary study

2 Lead UX Researchers (one based in Brazil and one abroad)

2 Junior UX Researchers (one based in Brazil and one abroad)

1 recruiter

Lead UX researcher

Moderator

Data analyst

Overall: 4+ weeks from start to finish of each project

Discovery & Research: 2+ weeks

Design & testing: 6 weeks

Session interviews: 3 days

Around 2 or 3 weeks before the fieldwork starts, the researcher in charge of the study gathers and shares the materials necessary for running the session, which may include:

Discussion guide

Mock-ups

Slide presentations

Figma files

Videos

These resources are often used to help and illustrate to participants the features the study will approach, or some features the participant should interact with.

Considering that rolling research happens in different countries at the same time, it's common to have discussion guides shared between the teams in different countries. Given the linguistic and social context of Brazil, it is up to us to adapt and translate the discussion guides to Brazilian standards and adapt and translate screens and mock-ups to the Brazilian technological context. We are always very careful not to go too far from the original wording of questions when adapting the discussion guide as this might affect the data we get from the study.

Although it is not our responsibility to build a discussion guide from scratch, one of our attributions as Brazilian-based researchers is to validate the discussion guide, as well as the questions and research hypotheses.

In addition to the translation, the adequacy of what is shown and discussed in each session is also our responsibility.

Based on socio-economic, age, education, and digital literacy; a sample is drawn up by recruiting people on social networks, online groups, forums, and lists.

The geographic criteria may include cities and specific regions that include users representing the majority of the Message App users in Brazil or representing a specific group, for example, WhatsApp Business users.

Following the recruiting criteria, the sessions are scheduled. Most of the rounds include 9 sessions with 60 to 90 minutes duration, depending on the topic covered. We usually recruit 1-2 extra people to make sure we always have a plan B in case we need backup.

In-depth interviews are the crux of the entire research process. At this time, we talk to users and seek to understand and respond to how our product meets or responds to their needs.

In-depth interviews provide an opportunity to understand the complex experiences of your users, including all their emotions, wishes, and difficulties. We make sure to let the participants be as comfortable as possible when sharing their views so that the topics can be discussed in more depth. During one interview (approx. 60-120 min) we can cover the entire customer journey and explore users’ experiences by using or interacting with the Message App features samples along the way. The interview can be supported by additional resources, such as images, note cards, or other devices.

All sessions take place remotely via Google Meet with simultaneous translation for stakeholders based in the UK and US, where we seek to understand the relationship of participants with the product.

To keep other teams and stakeholders updated while the research is ongoing, we schedule debrief calls with them and deliver debrief documents every day after fieldwork with a preview of our findings.

The results of all interviews are gathered in a datasheet, analyzed, and condensed into a topline report with the main insights, results, and recommendations highlighted along the document. Most often, we deliver the final report with all the information on the qualitative or quantitative research study performed that week within a couple of days after the interviews are fully completed.

The topline report is the main deliverable document shared once we condensed our research findings and insights to be used as a reference for product designers, and the Client's UX researchers. Based on this, it's possible to think about our next steps in the development of the Message App features as well as the next rounds of research until its definitive implementation.

An approach was adopted using print screens on each screen of the flow so that participants could closely observe a series of usability details that were tested during the sessions, i.e, the element's position on the screen, buttons, call-to-action for each screen tap and expected results for the following screens of the flow.

The research round process consists of developing a Minimum Viable Product (MVP) based on validated ideas and running tests to improve subsequent iterations. 

Creating prototypes is essential as they help you refine the final product, and they are here to make sure you’re on the right track. It is also easier, quicker, and cheaper to improve a prototype and run research once again to validate it than to redesign a ready-to-launch digital product.

It is important to highlight that receiving positive feedback through research is what we, UX Researchers, are always looking for when performing projects like the one above but that's not how the project ends for the Client's teams. The best product teams stay connected with their users to understand how they receive the feature once it's been launched and whether further improvements will be necessary. That's when performing rolling research as we do makes the biggest impact.

Researching products for the Message App is what puts us in an environment of continuous delivery, product improvement, and research optimization. We continuously improve our process to help execute the discovery and validation process of any product and/or service with efficiency, from recruitment to the choice of methodologies and tools to be used for good-quality findings and actionable insights.

Working in a fast-paced environment when researching for the Message App team gives me a sense of priority and urgency without forgetting to deliver reliable and robust results that will impact billions of users around the world and almost every citizen in Brazil today.

Learning to ask the right questions, gather the data, and deliver solutions quickly puts me and my team at a great advantage when it comes to delivering continuous improvements in strategic products like the Message App.

Let's share ideas & discuss ways to collaborate!