UX Research at Ethos

What is UX research?

User experience (UX) research drives product development by applying statistical and behavioral data. It incorporates quantitative and qualitative research methods to uncover user insights, which lead to concrete product changes.

Both quantitative and qualitative approaches are valuable. Quantitative methods such as surveys and usability test scoring are used to establish metrics and draw statistical inferences (i.e., does a certain attitude/behavior generalize to the overall user population?). We also leverage qualitative methods such as interviews and concept testing, which provide a more in-depth and open-ended understanding of our users.

How do we conduct research at Ethos?

We currently have three members on the Ethos research team, supporting consumer research (B2C), partnership/agents research (B2B), and marketing research. The stakeholders we most often work with include design, product management, and marketing.

When we partner with design, we most often work on usability testing. These are research sessions aimed at gathering feedback on the prototypes that designers work on. It is much cheaper (and faster) to conduct usability testing and identify things we can improve, make those changes in Figma or Sketch, before handing the final files over to the engineering team.

Note that we like to say usability testing, rather than user testing, because that is what these sessions are about — it’s to test the prototype, not to test the user. If a user cannot complete a task that the design was meant for, that doesn’t mean the user has failed. Rather, it means that there are improvements we can make in the design.

When we partner with product managers and the marketing team, we often do research that is more formative in nature. The findings on our users’ attitudes and behaviors help inform product and marketing strategy. These are projects aimed at reducing the shoulda/coulda/woulda — by integrating the voice of our users early and often, we increase the chance of creating something that will resonate with people.

Lessons We’ve Learned (1/3): Defining the right goals is key to productive research.

Defining appropriate project goals is key to aligning stakeholders and running a productive project. We’ve learned to be clear with the purpose of UX research, especially when a project request involves Key Performance Indicators (KPIs). Here are two examples:

• We want to increase conversion. Does design A or design B work better?
• We want to decrease early lapse. Does incentive A or incentive B work better?

It’s tricky to directly answer those questions through usability testing, especially when the nature of the test is hypothetical. When we show different designs in front of potential customers, as much as they may like or prefer a certain prototype, their affinity doesn’t guarantee conversion. They aren’t completing the purchase with actual money! Those questions are more suitable for real, live experiments. Nonetheless, research can add value. For example, we can:

• Reuse what we have: leverage past findings. Apply relevant design guidelines or recommendations that we have already identified
• Collect fresh feedback: get user feedback on new designs, identify areas of improvement, and make those changes before handoff

Lessons We’ve Learned (2/3): There’s no one-size-fits-all strategy for recruitment.

To run user research, we need participants. We currently have several recruitment routes:

1. Using the panels that come with our enterprise UX tools
2. Sending recruitment emails to customers who fit our recruitment criteria
3. Advertising research sessions for people to sign up on social media

Naturally, deciding which route to take depends on the project. If we’re looking to get input from actual customers, sending emails would be best because we’re directly reaching out to those who are guaranteed customers. If we go with platform panels or social media ads, there are additional steps we need to take in order to ensure that those who have signed up meet our criteria for recruitment.

One of the tough parts with recruitment centers around getting a sufficient sample size. Gathering a sufficient sample size helps reduce bias, so that we’re not making important decisions based on just a few individuals. This can mean interviewing more participants to gain a wider perspective or collecting more survey responses to assert statistical significance.

To boost participant sign-up rates, we’ve played with timing: for example, sending out a recruitment message at 10am vs 5pm. We’ve also experimented with incentives, like incentivizing every response but with a lower value (e.g., every participant who completes a survey receives a $10 gift card) vs raffling a higher-value gift card to one participant (e.g., we’ll raffle a $700 gift card). Last but not least, we tried imposing a time limit to see if the urgency would increase the response rate (e.g., “sign up today!”).

Related, since our mission is about making life insurance more accessible, it’s crucial to include groups that are typically less represented in user research. For example, in one project, we tried to recruit those without computer access via phone calls. We also made efforts to cover different subsets of our customer base by filtering based on demographics data, such as education level, geographic location, language, and age. Including different segments allows us to get additional perspectives on designing an accessible platform.

All in all, we’re still learning! There are other methods to try and we’re eager to define a process that works best depending on the research.

Lessons We’ve Learned (3/3): Sharing insights effectively takes practice.

A common challenge is boosting the visibility of our research insights, not only within stakeholders but also across teams. The research team at Ethos is relatively new and we have lots of opportunities to establish the practice. So far, we’ve done:

• Updates in Slack channels (product-specific or research-related)
• Smaller research presentations with stakeholder groups
• All-hands meeting research mentions via slide and short summary

Although these dissemination methods have sparked interesting conversations, we recognize that there is more we can do. For example, how might we make our findings more visible and actionable for those outside of our immediate product organization? We’re exploring other ideas, such as:

• Hosting larger meetings to share research results, such that whoever interested can join the discussion in real time
• Holding research Q&As and office hours, such that anyone can drop in and learn more
• Sending company-wide quarterly summaries of research insights, so that these materials are readily available for reference

Ultimately, making research findings accessible helps more Ethosaurs take actions based on user insights!

Wrapping up

In this blog post, we’ve highlighted three lessons:

1. Defining the right goals is key to productive research.
2. There’s no one-size-fits-all strategy for recruitment.
3. Sharing insights effectively takes practice.

Building products at Ethos has been an incredible learning experience. Interested in joining? We’re hiring! Learn more about our career opportunities here.

Rachel Chen joined Ethos in January, 2022 as a Senior UX Researcher. She is based in San Francisco and enjoys spending time with her dogs. Interested in joining Rachel’s team? Learn more about our career opportunities here.

Alexandra Folks joined Ethos in January, 2022 as a UX Researcher. She is based in Southern California and is a mama to her dog, Friday. Interested in joining Alexandra’s team? Learn more about our career opportunities here.