Objective alternatives to surveys in empirical research
By Murmuras on Sep 24, 2020
In both scientific and market research, surveys are among the top used data collection methods to get insights into human behavior. The questionnaire, as a research method, pioneered almost 200 years ago in London. While since then, the medium of conducting a survey has changed from paper-and-pen to online, the technique itself has essentially stayed the same. New technologies and neuroscience often offer more insightful ways to measure human behavior.
The problem of cognitive biases
Surveys always rely on the participants’ introspection. But the way participants see something is always influenced by their subjective opinions and experiences. Thus, researchers need to be aware that replies in a survey are not objective. Participants don’t passively reply to a question by itself but take a range of contextual factors into account. Most of these factors impact the response without the person even being aware of it. These processes are called cognitive biases.
Here are five of the most prevalent sources of biased responses:
Framing: The way a question is phrased and presented (framed) impacts how your participants reply to it.
Self-serving bias: The tendency of your participants to give answers that maintain and enhance their self-esteem.
Social desirability: Participants tend to give answers according to what they think is socially more acceptable and make them appear more likable than their ‘true’ answer.
Extreme response bias: Especially in surveys with Likert scales as answer format, participants may reply in extremes, even though they don’t truly hold this view.
Moderacy bias: Contrary to an extreme response style, people can tend to choose medium answers overly.
An additional problem with surveys is that they can only be administered to a person once or, at most, a few specific moments in time. Even asking participants to fill in questionnaires several times only gives momentary snapshot and not continuous insights. The dangers connected to this are asking people to report what they did or felt like, in retrospect, answers are highly inaccurate as cognitive biases distort memory. Additionally, what mood participants are in when replying to a survey impacts their responses. So even answers of the same person at two different points in time are not necessarily the same.
Furthermore, online surveys also tend to have high dropout rates, especially if they take up a long time or are administered several times. Having many dropouts can be problematic as it makes it harder to get enough data and can take a long time.
Alternatives to surveys
There are further “traditional methods” to study human behavior, such as experiments and interviews, but these methods also rely on self-reports and suffer from similar disadvantages as surveys. Yet, in times of Big Data, AI, and modern health technologies, researchers can pick from more objective alternatives.
One way to investigate what’s going on in people’s heads is to use measures such as eye-trackers, EEG, or fMRI. Other physiological indices, such as a rise in stress hormones or heart rate, can also be insightful. Unfortunately, most of these methods need special equipment and training.
Another more accessible alternative to study behavior is to analyze smartphone data. As constant companions, smartphones provide objective access to people’s daily lives and behavior. They capture both the digital behavior (i.e., daily routines, social media habits, communication behavior) and real-world movement (i.e., GPS). Furthermore, wearables can be connected to get objective health data.
Though not offering a view of participants’ brains directly, modern objective research methods provide a realistic picture of people’s daily behavior in real-time. Based on the research question, using such an objective measure in addition to one of the more traditional methods can also help to validate the responses of the latter. For most research question these tools add great value to better understand human behavior.