How weekly tests grew our active users 110%

Denislav Jeliazkov

UX Designer

Product Designer

UI Designer

Figma

Google Docs

Principle

The project encompassed tasks related to User Experience, Interaction design, and Interface design.

In today’s era, each of our choices is supported by data. The technology at our disposal possesses a deeper understanding of our preferences than we may have.

Before investing significantly in a new marketing initiative or introducing a product feature, we now can scrutinize data that will decisively steer us toward making sound business decisions.

Problem exploration

As a designer who isn’t technical, understanding the entirety of the product was crucial. Without any pre-existing insights, I started by picking up pieces of the experience and reverse-engineering it.

The initial pain points were clear to me, after talking with dozens of users. For people who don’t have experience in this, it was tough to:

Objective

To aid new users in becoming more familiar with how to structure and get started on their first pipeline without becoming immediately overwhelmed with the available options in each step.

Success would be measured with an NPS survey and by tracking the people who complete their first pipeline on the first try.

Our starting point wasn’t as good as we expected, especially since most of our users were experienced, users. We sent them a survey to see how many of them are promoters, passive or detractors.

As you can see, not good.

Non-linear process

Usually, when tasked with a problem I don’t follow a strict process. I prefer before moving forward to double-check and fix any inconsistencies in my statements before proceeding to the next steps, for a product as complicated as this it was extremely hard to predict the architecture. This way I set a better problem definition and improve the solution in every iteration.

Gaining a deep understanding

I was not able to learn much by conducting secondary research because it’s a very specific topic and my time was limited, so I stepped forward to the primary research.

User interviews

I made a list of potential participants and conducted a short interview, to uncover more about how they build Pipelines and where they struggle at the moment. My main goal with this was to understand how they plan and execute the build process.

I picked 11 people (7 customers and 4 internal team members).

With the intent to perform the task of building a fairly simple experience and ask them questions along the way, we were able to uncover a lot of their pain points and frustration which helped me form a set of problems and interaction patterns I’d need to observe and focus on in order to have a delightful build experience.

Learn and understand users struggle with creating a new Pipeline

Understand how they plan to create a Pipeline

Do they know in what order to set a Pipeline?

Test assumptions of simplified workflow

Does a more guided wizard creation flow help you speed up the process or not?

Learn about areas where they feel lost in the process

Understand the areas where they feel the product can be improved

Understand how they plan, organize, and execute creating a Pipeline

Additional possibilities

Get a general idea of how easy it is for them to work within the steps, connect to their apps, and organize the pipeline.

Initial research results

I synthesized and cataloged the needs and the problems that I uncovered.

Most prominent pain points

It’s not a surprise that people aren’t completing their pipelines, as the majority of them didn’t make a difference between Trigger, Scheduled, or Manual.

For novice users, it was scary that they are starting from a blank canvas and know what they want to achieve, but don’t know how to achieve it.

One of the unexpected insights was that people were confused about the “JINJA” expression, as it’s displayed in many fields.

When people made a mistake, there wasn’t a undo of any sort, and the only way to fix an error was to delete a step and add it with the correct information.

General insights

People expected they could reorder steps by dragging, which worked to an extent but was extremely clunky.

The code base was outdated, and the product was prolonged.

Collaborating with teammates was hard to navigate inside another person’s build.

Setting the initial problem hypothesis

People wanted to have a more guided flow to build experiences and a product that was more forgiving of mistakes and actually helped users if they made a mistake.

Competitive apps exploration

Our users were familiar with other products that have the same capabilities. Several of them mentioned Workato, Automator, and Zapier. Most people are also quite familiar with Microsoft Automate as it fits their organization and suite of tools, making it easier.

Initial Flow Exploration

After the competitive research, I set out to map a general journey to create your initial two-step pipeline, as after looking at some data, most pipelines were with three or fewer steps.

My goal was to create a few scenarios and use cases to achieve the same result — build a pipeline faster.

Start with a template — One where you start with a template, which is simplified by saying that you jump directly into the builder with two steps, but thinking what the end result will be.

Wizard — Our initial exploration was to give people more freedom to create pipelines the way they wanted. So we left them with very little guidance but also provided them with enough information.

Testing initial flows

After mapping out the initial flow, it was time to test it. We set out for the same group of 11 people we tried before and presented them with a clickable prototype.

We set up moderated user testing, where we presented them with a clickable prototype for them to create a pipeline.

The biggest findings of the research were:

Iterating?

After our research, we understood that although the experience is more guided and easier to build, it still wasn’t solving one of the critical problems users have — how do you make a working pipeline?

With that in mind, I explored different solutions to the build experience after people have left the initial wizard.

One of the things we had to solve for is how do we make the build experience more guided. After a few iterations, I felt we needed to test a step-by-step approach where it showed you only one action or task at a time.

After creating a prototype, we set for an unmoderated user test and presented the user with a clickable prototype to create a simple two-step pipeline where in the first step, they had to select all fields.

The biggest findings of the research were:

Interaction enhancements

A lot of the work I’ve done during this time was focused on micro-interactions and how subtle elements work in this complex web for experiences.

Some of them deserve a case study on their own, but I won’t do that. Instead, I will list them here so people can understand how much work goes into something that looks as simple as “a few fields.”

Adding a filter to a step

Besides the conditions you can set on a workflow, we can add a condition to criteria and parameters to a step field.

Select or type values

One of the main problems is that the previous selector was floating.

Since some of the values are static to a field and others are dynamic, which you can recall from a previous step, we set out to figure out a way to display them both in a single field to make it easier to contextualize.

Writing custom code

One of the significant benefits of pipelines is the ability to write custom code.

This is a more advanced feature, but we wanted people to be able to write a custom expression if they didn't find a match in the predefined fields or the previous step.

Conditions

Adding logic and queries is one of the biggest parts of the product.

My goal after the rounds of research we did was to make it as easy as possible for people to view conditions and different paths of the workflow so they can think about what to build next as a step.

Outcomes

After our initial beta build of the experience, we sent an NPS survey to newer users and all of the previous users who answered the survey.

The results speak for themselves, and although the survey didn’t get as many responses as we wanted, we quickly understood we were on the right track.

Our active users grew from 6000 to over 14000! Put it simply, more than 110%.

This wouldn’t have been possible without the combined effort of myself, the dev team, and our PM.

Limitations

As with any project, there were certain limitations I had to deal with. As a product company, I had to work with an established visual design language. Although this was a limitation, it was also an opportunity for our design team to see what we could improve and update.

Take-Aways

In the course of working on this, I had the privilege to learn more engineering-focused terms and workflows which I haven’t faced before. This project taught me even more about the importance of teamwork and a user-centered process.

I share some tips on design and behind-the-scenes of my process on Instagram and Twitter

Like this project

In today’s era, each of our choices is supported by data. The technology at our disposal possesses a deeper understanding of our preferences than we may have. …

Likes

Views

Clients

Quickbase