Avoiding Research Bias… Everybody Lies!
If we could understand consumers’ biases, we could create deeper, more credible recommendations. The problem with avoiding Research Bias is… Everybody lies. All the time.
Need examples? Sure.
We lie about voting – whether we will or not, and who for
We lie about sex – how much we’re having, who with and how
We lie about brands and products – what we do and will buy and use
Deliberate vs Unintentional lies
Sometimes these lies are deliberate. We call this form of Research Bias a social bias, meaning we want to project a certain image of ourselves to an interviewer, a moderator or the outside world at large. Often these lies are unintentional. We know what we’re asked, but we can’t accurately recall former behaviour. Think about the question ‘What beer did you drink 4 weeks ago? It’s quite possible you think it’s Heineken, but it could have also been Asahi. Other times researchers instill Research Bias by asking the ‘wrong’ questions’. Lastly, often respondents don’t really understand how they made, make or will make their decisions. Often we don’t even know what we truly think or feel about many topics.
How is this a new insight?
Of course this is nothing new. We use various tools, techniques, applications, gamification and so on to avoid Research Bias… as much as we can! We have worked with databases, benchmarks, adjustment factors, rules of thumb – all attempts to ‘correct’ for the biases we know of. Qualitatively we use extensive projection and elicitation techniques to try to get people to reveal their true feelings.
But it’s only partially working. The Research Industry is at a cross-roads. As a matter of fact we all are. IF we persist in ignoring Research Bias we will soon be stuck somewhere between ‘having credibility issues’ and being a complete laughing stock. Yet still, the vast majority of the research and insights we work with are straight reporting of claimed ‘facts’. Still, most of the reports I see (internal and external), and almost all of the initial proposals I receive, are based almost exclusively on single-source research.
What should we do?
Seth Stephens-Davidowitz has written an excellent book about Research Bias titled Everybody Lies. He goes quite far in some of his observations and claims. It also isn’t necessary to agree with all that he claims… but it does give an interesting perspective and calls upon us researchers to ‘do better’. Combining multiple sources of data, have qualitative research complement quantitative research (and vice versa).
I would advise you to also read the book. With an open mind.
Build your tool-box. Become familiar with the new approaches that are available. See how easy it is to create your own insights.
Resolve to add behavioural dimensions to reports. If you are proposing a new innovation space based on research, demonstrate how search terms around that space are increasing. If you are writing a proposal for communications tracking, insist on getting access to View to Completion rates across digital channels to compliment or challenge your findings.
Everybody lies. All the time. We’ve always known it. It’s about time we took more initiative to address it directly.