Understanding Survey Response Rates for Better Market Research Insights

Monitor response rates to ensure your sample is representative and that you are collecting quality responses.

Even experienced researchers sometimes overlook the importance of survey response rates. Some of them might say, “As long as I get my sample quota, why do I have to worry?” But there is much more to response rate than simply getting respondents through your survey. In fact, a low survey response rate could be an indication of problems that can directly impact the quality of your study data.

I. Why Response Rates Matter

The response rate is the percentage of people who complete your survey out of the number of potential participants contacted. A high (or “acceptable”) study response rate is important to ensure your results are representative of your target sample and that your questionnaire is performing as intended. As Xiaoyi Chai, dataSpring General Manager, China points out, “A low survey acceptance rate or high questionnaire dropout rate can be an indication of potential issues with the study.”

II. Acceptable Response Rates

Industry standards for participation are important to consider. Rates vary depending on methodology, but your study should be within the following ranges to ensure acceptable sample representation:

Online: 30%
Phone: 80%
Mail: 50%
In-person: 80%

There are a number of things to consider when setting up your study to help put these numbers into context:

A. Research Objective

If the purpose of your study is to project results to a larger population (Awareness and Usage, Product Testing, Polling, Segmentation), response rates are critical. If the study is more exploratory (mobile ethnography) or seeking general insights about attitudes or product category insights, representation is less important so slightly lower response rates are less of a concern.

B. Analysis Plan

When looking to compare results across measures (for example in a product evaluation test), minimum sample sizes are required to determine significance. A lower than expected response rate may result in having too small a sample to perform significance testing or other statistical analysis.

This is why it is important to share your analysis plan with your sample provider to ensure an adequate number of surveys are completed.

Factors that Impact Response

Download Mobile 101: A Primer for Researchers

III. Factors that Impact Response

To help ensure acceptable response rates, think through the following factors and adjust your study approach and sample plan accordingly:

A. Respondent Population

The range of potential respondents for a study varies by sampling approach. From what would be considered a random sample (RDD phone studies) to panel samples, response rates are generally better when potential respondents feel like the company, brand or group conducting the study is reputable and trustworthy. This is why it is important to use a quality sample provider. In the case of a panel sample, a quality sample firm will have already established trust and a bond with potential respondents, generally leading to a stronger response to your survey.

B. Study Invitation

This is crucial for both online and offline methods. A good invite should not only motivate participation, but equally important, set clear and reasonable expectations for the requirements to complete the study (length of time for completing the survey, etc.). Remember that the survey response rate is based on completed surveys; getting people to start the questionnaire only to drop out results in a lower response rate.

C. Survey Topic

Some topics are going to be of more interest to potential participants than others. Additionally, some respondents may decline participation if the topic is considered potentially “sensitive” or too personal. For sensitive topics, be as reassuring as possible in terms of confidentiality of the data. In cases where the data will be used to address an issue (such as a medical condition or personal care problem), it can help to note that the information provided will be beneficial to other sufferers of the condition or problem.

D. Survey Design

A well-worded invitation can bring lots of potential respondents into your survey taking process, but a poorly designed study can lead to significant dropouts and unacceptable response rates. Be mindful of:

a. Survey Length:

Do not assume respondents will give you unlimited amounts of time to go through your survey. Attention spans are declining (see the article Humans have shorter attention spans than goldfish, thanks to cell phones) so long questionnaires will lead to high dropout rates or potentially worse, respondents not paying full attention to their answers; either of these leads to potentially compromised data quality.

b. Survey Logic:

Make sure there are flow and logic between questions. Disjointed questionnaires leave respondents frustrated, raising the perceived length of the study and leading to higher dropout rates. Be sure to go through your study (and have others go through it as well) to make sure it makes sense and is easy to follow.

c. Question Wording:

As with survey logic, poorly worded questions and response options lead to respondent frustration and higher dropout rates. It also degrades data quality if different respondents are interpreting questions differently. In addition to crafting short and concise questions, eliminate jargon and/or local idioms. This is especially important in survey translation and multi-region studies.

E. Incentives

One of the more important drivers of response rate is the inducement (either monetary or otherwise) used to motivate people to take a survey. Tailoring incentives to the target (or region) is an important consideration. This is another area where using an experienced sample provider is invaluable. A quality firm will know what type of incentives best motivate participation. Some groups, for example, prefer purely monetary rewards, others are motivated by accumulating points, while others prefer tangible items. Another thing a quality sample firm will provide is timely and efficient incentive fulfillment.

Final Thoughts

Learn about dataSpring's Mobile Capabilities