How to screen panels for valid survey responses

Survey screening questions are used to identify the respondents who are most relevant to your research. Generally, research panel platforms profile participants based largely on demographic criteria. Survey screening questions, also called "screeners", go beyond general demographics and use more specific criteria to qualify participants.

Survey screening questions are asked before the actual survey begins. This way, you can ensure the right audience is responding to your survey.

Let's say you've designed a new summer program for your school and you want to know how many parents will enroll their children. You'll need to make sure your survey is reaching respondents with children in the appropriate age range, with the appropriate interests. To accomplish this, you can add screening questions like, "Do you have children under the age of 15?" or "Which extracurricular activities is your child interested in?"

Screening categories

Screening questions based on advanced demographic requirements

Demographic targeting is one of the main ways you can filter respondents. You may want to distribute your survey exclusively to people belonging to a specific age group, gender, or geographical region. There are many tools available to help you target based on demographics. But if you can't find the right tools to reach a niche demographic, adding screening questions is a useful solution.

For example, you can target pet owners with the following screening question:

"How many pets have you adopted/fostered/bought?"

0
1
2
3
More than 3

Screening questions based on behavior and interests

Behavior and interest targeting refers to selecting respondents based on their buying habits or interests.

"How often do you watch content on OTT platforms?"

Several times per day
Once per day
Few times per week
Few times per month
Hardly ever watch
Never watch

If you're conducting research about people's experiences with OTT media platforms, it would be wise to exclude participants who "hardly ever watch" or "never watch" OTT content.

Screening questions based on industry

Targeting specific industries for research has been on the rise. Respondents in specific industries can provide valuable insights for your research. In many cases, you'll collect more relevant data from within a specific industry than you will by casting a wide net over the general population.

For example, while doing research on pharmaceuticals, your preferred respondents will likely work in the healthcare industry. Experts in this field will be in a unique position to provide you with valuable and relevant feedback.

Why are screening questions essential?

Save money

When done right, survey screening questions help you reduce the overall cost of buying responses from an audience panel. When an unscreened person answers a survey about soft drink preferences, even if they don't consume them, you end up paying for the response. It's far more cost effective to screen out respondents who aren't relevant to your survey.

Save time

Without screening questions, you're likely to waste time wading through poor quality data to get the information you need. Save time by ensuring your responses come from participants who meet your criteria and offer relevant experience.

Ensure quality data

When targeting fails, screeners can be used as a safety net, to ensure only your target respondents participate. With the right participants, you'll receive a lower percentage of unusable data, leaving you to work with clean and relevant information.

Calculate incidence rates

Screening questions help you determine the incidence rate for your research panel. An incidence rate is the percentage of respondents you expect to qualify for the survey.

Let's say you require 500 survey responses, but estimate that only 50% of your audience will be eligible to answer your survey. In that case, 50% is your incidence rate and you will need a sample size of 1,000. The lower the incidence rate, the higher your sample size needs to be in order to get the required number of legitimate responses.

With survey screeners in place, you can estimate a higher incidence rate, and spend less money on your sample size.

Here are some tips for designing screening questions with Zoho Survey

To get the right participants, you have to ask your screening questions the right way. Here are some helpful tips:

Keep screening questions separate from the rest of your survey

Ask your screening questions on a separate page at the beginning of the survey. This way, you can guide disqualified participants to the exit page. With Zoho Survey, you can display a customized message informing participants that they do not qualify for your research.

Don't reveal the topic of your research

If the respondent knows the research topic before answering the screening questions, they might try to provide the "right" answer, instead of a truthful answer. Keep your topic secret to eliminate ineligible respondents who want to participate for the money or rewards offered.

For example, if the survey is about preferred car insurance schemes, a screening question can be:

What vehicle do you use?
a) Car
b) Bike
c) I take public transport

Use the right question types

For questions that require multiple selections, don't use a question format that restricts respondents to one answer. This would eliminate viable respondents.

Make it mandatory

As a general rule, it's best not to make questions in your survey mandatory. Participants should have the choice to skip questions that are not relevant to them. With survey screeners, it's the exact opposite. Make your screening questions mandatory and apply restrictions and option expectations.

Avoid leading questions and biases

Don't phrase the question in a way that might sway respondents towards a particular answer. Your biases may influence their responses.

Instead of asking, "How often do you keep yourself healthy by exercising?" consider asking, "Is exercise a part of your daily routine?" Give answer choices that range from the negative to the positive.

Randomize options

Survey participants that tend to straight-line or speed through questions can be easily detected with the help of randomizing answer choices. Include two or three similar questions, and if the participant answers them inconsistently, you can disqualify them.

Avoid jargon

Using industry-specific terms and difficult-to-understand language may confuse your participants. If they don't understand the question, it will be difficult for them to choose an accurate response.

Avoid yes/no questions

When you have "yes" or "no" answer choices, a respondent can quickly guess which answer they should opt for in order to proceed with the survey, and get the promised reward.

Run a test batch

Invite reviewers to check out your survey and check for any possible errors and loopholes in your screening questions.