Non-response is the greatest threat to voluntary survey result accuracy. Different surveys obtain dissimilar response rates depending on several factors. For example, the surveys that ask relevant and intriguing questions to respondents obtain the highest response rates. Unfortunately, in the recent years response rates of popular surveys are declining. Therefore, there is a concern about increased survey bias.
The nonresponse is an issue only when the non-respondents belong to a non-random sampling portion of the total sample. For instance, in household surveys, there is evidence that the non-respondents are younger than the respondents. Men are hard to persuade for answering survey questions in comparison to women. In cities and deprived regions, the response rates are lower than average.
The outcome of these surveys will not echo the population views they are chosen to represent. Typically, surveys overrepresent females over 30 years. Surveys often underrepresent respondents residing in deprived regions and cities. To avoid such a poor match between population and sample, the weighting technique is used to bring both closely in synch. It is also termed as ‘Non-response weighting’.
Weighting data is a bridge to reconciling minor nonresponse bias. You can learn more about it from the marketing research experts at OvationMR.
Non-response adjustment using the post stratification
- The first step is to determine a total population that the survey needs to match.
- The second step is to calculate weights and adjust sample totals with the total population.
For example –
Men 40 -50 15.0
Women 40-50 16.0
Men 40-50 14.0
Women 40-50 16.8
To poststratify the sample, calculate the weights to bring sample distribution in alignment with the population. So, the weight applied in –
- Men 40-50 is 15.0/14.0
- Women 40-50 is 16.0/16.8
To use post-stratification there is a need to know population distributions. The total population that the survey needs to match is only the ones available and legitimate.
Post-stratification compares the n-way table for the total population to the equivalent n-way table for a sample. Weight is calculated per cell to adjust the sample to the total population. If there is no population number available but marginal distributions are known then methods like raking and calibration are employed.
In raking it is assumed that non-respondents are similar to respondents. It enhances the mean squared error in the sample estimates. The calibration uses an iterative procedure for household surveys. The underlying hypothesis is that non-response is basically a household decision instead of a personal decision.
Nonresponse at households creates major discrepancies between population distribution and sample. A regression model is used when plenty of data from the population is available. The post-stratification cells seem small and can cause sampling error. However, using the regression model the weights are smoothened to attain more stable estimates.
Nonresponse weighting using a sampling frame
Compare the respondent’s characteristics with the group that did not respond to the survey. It may not be possible if you are not aware of the characteristics of the non-responsive group. It is an effective approach when you choose a sample from an informative sampling frame.
For example, in an employee survey, the researchers are aware of the age, experience, and grade of every employee. Sometimes the sampling frame has more detailed information associated with non-responders. Therefore, logistic regression is used to investigate which factors influenced the non-response.