Any study that collects and relies on self-reported attitude, usage, or behaviors is subject to some amount of over or under reporting bias. Find out how to combat these issues in this blog post.

In other words, respondents may not always be able to accurately recall all the things we as researchers and consultants would like to know. In some cases, this type of bias is genuinely linked to not being able to accurately recall an occasion or use. But other times, the bias in self-reported data is linked to a respondent’s own self-perception of social desirability and status. This is true in many Asian countries.

“The rapid economic rise of certain population groups and the growth of more distinct social classes have made social desirability bias in market research more of an issue,” says Irene Wang, dataSpring’s Sales Director in China. “It is crucial to recognize and address this issue in our study designs so it does not compromise data quality.”

Affect on Data Quality

Social desirability bias can affect data quality and the validity of insights in a number of ways. One way is if respondents falsely claim to own, use or purchase a brand. This is especially true when researching higher-end product categories such as consumer electronics, automobiles, certain retail brands, and beauty/cosmetics/fragrances. In this case, the respondent is trying to identify with an aspirational brand to raise their own perception of their status and social desirability. This can have major effects on data integrity, especially when trying to get feedback on the importance and satisfaction with key brand/product attributes. In other words, if a respondent does not, in fact, own or use a product, their input is meaningless. It can also affect measures of future intention by inflating some projections while deflating others. For example, a common question among owners/users of a brand or product is “How likely are you to purchase this product in the future” or “How likely are you to recommend this product to a friend.” Again, there is no value to responses to these measures from someone who does not own or use the product.

Ways to Combat Social Desirability Bias

1. Sample

First and foremost, it is critical to use a quality sample supplier to provide valid and properly screened respondents. Sample houses that lack good panel management and do not qualify, regularly rescreen and purge panelists are more likely to let suspect respondents into a study. High-quality panelists understand the importance of honesty, and reputable sample providers continually reinforce this. Panelists recruited by pop-up sample providers often attempt to adjust their responses to qualify for as many studies as possible, which can negatively affect data quality and the validity of insights

2. Questionnaire Design

A properly design questionnaire is key to combating social desirability bias. A good screener will not tip off the respondent as to what criteria will qualify for a study, while a well-crafted question will not bias the response.

For example, if looking for respondents who participate in a certain activity or purchase behavior, it is important to start broad then narrow things down.

When owning/using a certain brand is the criteria, utilize unaided responses instead of brand lists.

When dealing with things like income, avoid binary (above/below, yes/no) questions.

By properly crafting your screener you will get the right respondents into your market research survey, helping to eliminate bias based on characteristics and behaviors that are not the respondent’s own.

3. Supplier Choice

Experienced researchers and consultants should feel comfortable drafting their own screeners and questionnaires, but having a seasoned and experienced sample or research supplier review the survey is well worth the effort. Sample providers and suppliers see thousands of studies a year and can help identify questions that may be more likely to induce social desirability bias. This is especially true in multi-country studies. In choosing a sample firm for global research, look to a provider with experience and a presence in the region.

Wrapping it Up

While most panelists respond genuinely and honestly when completing a survey, in some situations social desirability bias can creep into a study and affect the data. By utilizing a quality sample provider like dataSpring, properly drafting the screener and questionnaire, and utilizing the experience of your supplier, you can help eliminate this issue.

For more information on how to design your questionnaires properly, check out our market research survey essentials, or download our online survey terms basics brochure.