Know Your Users, But How Much Should You Trust Them?
By Lulu Xiao, Senior Strategist
I am a zealous advocate of design thinking and a passionate believer of user research. I love testing with users and listening to their feedback. I can’t help but get jazzed anytime I get to learn from them.
However, I rarely take what research participants tell me at face value. That sounds counter-intuitive, especially in the user-centered design world we live in, but hear me out.
I don’t take what research participants tell me at face value, because they are…well, humans. So, by virtue, they are – like all of us – mysterious, complicated individuals. Consequently, it’s important to watch out for certain human pitfalls and be incredibly diligent, conscientious, and even questioning when you’re learning about users. To truly comprehend users, you need to apply some critical thinking in your analysis and evaluation. It’s dangerous to do otherwise.
Specifically, it can be challenging to understand user research participants because:
1. Like all humans, what they say is not always what they do (or will do).
It’s understandable, and even expected, that participants experience confirmation bias when evaluating products or features and feel a need to be overly lavish with praise. Though it’s more prevalent in in-person interviews, I also see it in anonymous surveys and results from other user research methods. Even the most “honest” participants will couch their negative reactions in reassuring and positive words.
Moreover, though a participant might say that they want to buy a product straightaway, we simply have no way to guarantee that they will actually purchase it the next time they see it in a store, whatever the reason might be. It’s like going to the gym. There have certainly been times when I’ve said that I was going to start working out in the mornings…but come 5am and it’s warm and cozy in bed, and my eyes still feel tired…that’s a different story. Our actions don’t always mirror what we say. It’s just part of being human.
2. Like all humans, they are full of contradictions and are sometimes irrational.
I’ve seen this happen many times in user testing: a participant will struggle to interact with a product or feature, but when asked on a five-point Likert Scale how easy or difficult it was to complete the given task, he/she answers “very easy”. No one likes admitting that they had difficulty with something, so that may be at play here, but the situation is equally common in concept testing. A participant will explain that a product isn’t very interesting or useful, but then say he/she would be very likely to purchase the product. The reverse situation also occurs. All the scenarios reveal absolute contradictions. Sometimes, the participant’s explanation of his/her rating will clear up the confusion; other times, you’re still left with conflicting statements.
Along the same lines, I’ve heard participants declare that they would pay an extravagant amount for a product that they were either ambivalent about or disliked. Though they are not wrong, their reasoning is objectively irrational, or at the very least, inconsistent with what you would expect given their level of attraction to the product.
3. Lastly, like all humans, they don’t always know what they want
As Henry Ford said, “If I had asked people what they wanted, they would have said faster horses.” (As a strategist & researcher, it’s my prerogative to admit that the evidence doesn’t confirm Ford actually said this line. However, the point still stands.) People don’t always recognize what it is that will solve their problems or make their lives easier. There are many possible reasons for this. It may be because they haven’t had a chance to fully reflect on all facets of their problem(s), for example, or because they haven’t determined the best solution that addresses their problem(s).
Especially in a testing scenario, it’s challenging for research participants to identify solutions and articulate creative product/feature ideas on the spot. Though our final products and solutions frequently have flavors of what research participants propose, it’s unusual for them to be exactly what research participants recommended.
Users are of the utmost importance to designing the best products and solutions. We know how dangerous it is to neglect getting users’ direct input and feedback in the design process. It’s what bridges the gap to solution generation, and what can make the difference between a product that is successful and one that flops.
Yet, at the end of the day, user research participants are still humans – unruly, complex beings. So, it’s up to us to apply our own human analysis skills and understand what they really mean or want to convey.
Fortunately, this is one of my favorite parts of my role! I must interpret what participants say – or don’t say – so that the larger team has a comprehensive, deep understanding of user pain points, values, wants, and needs, and ensure we stay true to our users. I must decide what kind of information from research participants to trust at face value and what must be further deciphered, considered against past behavior, triangulated with other data points, or even battle-tested in the wild. I trust that everything research participants tell me is sincere, genuine, and legitimate. But I still seek out every opportunity to question and confirm what we have heard and know.
That’s how you really get to know your users.