How to combine quantitative and qualitative user research

Do you work primarily with quantitative user research? Or are you more about the qualitative user research? Either way, how do you best combine the two? Both types of research require a different set of expertise and mindset and mastering or even just working with both can be a real challenge. Oftentimes, it's only by combining quantitative and qualitative user research that you can get the best results.

Difference between quantitative & qualitative research

Quantitative user research is all about collecting data on user behavior, which usually is a numbers-game that requires the much-hailed 'data driven mindset'. Quantitative data is by its very nature concrete and objective and mainly answers questions around what users are doing, how many of them are doing it and how often they do it.

Qualitative user research on the other hand focuses on the experience of users to identify underlying reasons for their behavior, which usually is heavily used within the design thinking process. The results of qualitative research more resemble quotes and expressions, which is more abstract, and subjective. Qualitative data tells us why and how users doing something and gives insight into how you could improve your product or service.

Despite the differences between both methods, one is not better than the other. They answer different questions and most of the time, you will need to combine both qualitative and quantitative research to answer certain questions.

Combining qualitative and quantitative data

Quantitative and qualitative research belong together. When combined, they show you what your users are doing and why they are doing it. That’s a solid foundation for making decisions on product improvements and innovation. Focusing on only one type of data is a common mistake professionals make, leaving them with either limited context or at risk of biases.

Imagine you have a web shop and you have a high percentage of dropouts in the final step of the checkout process. You immediately search for ways to improve the checkout flow: maybe the delivery costs are too high, people need another payment option or they’re afraid that they won’t get their products (on time). Just to be sure, you try to fix all these assumptions at once and hope for the best. Unfortunately, the high percentage of dropouts remain. Another team conducts a usability test and discovers that the discount for certain items in the web shop isn’t shown in the checkout. This gives the users the idea that they miss out on a deal and are not willing to buy the item any longer. After fixing this problem, the dropouts decrease significantly.

Loaded with numbers and the stories behind those numbers, you can make better (design) decisions because you know the what, how (much/many), and why of certain user behaviors. This will not only help you improve your products and services but also to come up with valuable ideas for the future.

Ways of combining quantitative with qualitative research

Okay, we’ve talked enough about the value of combining both research types. I hope you’re convinced now and you want to get started. There are four ways for designing a research study with both quantitative and qualitative methods: Triangulation Design, Embedded Design, Explanatory Design, and Exploratory Design.

#1 Triangulation Design

You conduct a quantitative and qualitative study at the same time and the results of each study are equally weighed in the data analysis.

Use: for all complex research questions such as how to improve or innovate your product.

• Most time-efficient and effective

• Offers a complete picture because you discover the what, how (much / many) and the why

• Requires good teamwork and some extra effort for the researchers of the quantitative study and of the qualitative study to be able to merge the data for the analysis or interpretation

Example of triangulation design for user research:

You would like to know how people experience the homepage of your product or service. So you do an online survey (quantitative) and interviews (qualitative). With the online survey you learn how many people have a positive or negative experience with the homepage. The interviews show why that’s the case.

#2 Embedded Design

A quantitative study that is supported with data of a qualitative study or vice versa to answer the research question(s). The data of the supportive study provides a secondary role during the analysis or interpretation.

Use: if the secondary data type is only useful or meaningful if it’s embedded in the other data set.

• Cost less time and resources than conducting a quantitative and qualitative study in sequence

• It’s easier to embed one type of data within the other type than to merge them (like triangulation design) for the analysis or interpretation

• It can be difficult to determine when and how to embed the secondary data type for answering the research question(s) accordingly

Example of embedded design for user research:

The return process of your product could be improved. So you measure the duration and success rate of each step (quantitative) in the return process and ask people to give you feedback (qualitative) when they have returned their product. This way you know which step in the return process is going slow and/or wrong and which events in the return process influence people’s experience the most.

#3 Explanatory Design

Use: if you don’t know the cause of your quantitative results. This mixed-method research design usually is applied spontaneously if quantitative studies have surprising or unexplainable results.

• You have two studies you can also use separately

• It’s easier to conduct and analyze than Triangulation Design or Embedded Design because you conduct and analyse the studies separately and connect the results of both studies at the end, instead of conducting and analyzing two types of research at the same time.

• The duration of the research is longer than Triangulation Design or Embedded Design because you cannot perform the two different research types simultaneously.

The example about the check-out process at the beginning of this article, is an illustration of Explanatory Design.

#4 Exploratory Design

Exploratory Design is the reverse of explanatory design: you gather qualitative insights and quantify them afterwards.

Use: when you’re not able to measure things, the variables are unknown or you want to explore a certain topic or problem in depth.

It has the same pros & cons as Explanatory Design.

Example of exploratory design for user research:

The target group of your product is too broad. You want to know if you can split it into different segments and how big these segments are. Therefore, you interview 12 different types of users (qualitative) and discover some variables that are related to your product. With these variables you send a survey (quantitative) to your users to find out if you can discover patterns and create user segments from it.

Team up — you don’t have to do it all by yourself

Team up if you want to start combining quantitative and qualitative research! Most likely, you’re specialized in quantitative research OR qualitative research, so find someone in your organization with the other expertise to help you out.

To get the most out of your quantitative and qualitative research it’s important to have one place where everyone in your organization can gather, share, and organize the learnings from research and feedback. It helps a lot to know what others have discovered and what they are planning to do so that all teams are up-to-date with the latest insights/studies and can enrich each other’s work.

Don’t have a tool to centralize insights yet? Try NEXT and make better sense of your customer insights and data.‍

Sources

1. Creswell, J. W. (2006, September 26). Designing and Conducting Mixed Methods Research.

2. Rohrer, C. (2014, October 12). When to Use Which User-Experience Research Methods. <https://www.nngroup.com/articles/which-ux-research-methods/>