Nonresponse Bias: Definition, Examples, and How to Detect It

Nonresponse bias occurs when the people who don't take part in a study differ from the people who do, in ways that affect the results. It's also called non-response bias or nonrespondent bias. You invite a sample, some people never answer, and the ones who answer no longer represent the group you meant to study. That gap can shift your estimates in one direction, and a larger sample won't close it.

Nonresponse bias is a selection problem, not a measurement problem. It's about who is missing from your data, not whether the answers you received are accurate. That second issue is response bias, which has its own causes and fixes. This guide explains what nonresponse bias is and why response rates can mislead you. It also covers how to detect the bias, how to reduce it, and how to report it so reviewers can judge your findings.

Nonresponse Bias Detection Methods at a Glance

Each method answers a different question, and none is complete on its own. Strong studies usually combine two or more.

What Is Nonresponse Bias?

Nonresponse bias is the difference between a study's results and the results it would have produced if everyone sampled had responded. It arises when the reasons people don't respond are linked to the variables you're measuring. If those reasons are unrelated to your outcome, nonresponse shrinks your sample but doesn't bias your estimates.

Nonresponse is the behavior, and nonresponse bias is one possible consequence. A survey with high nonresponse can have little bias on one question and serious bias on another. That's because the bias belongs to each estimate, not to the survey as a whole.

Unit nonresponse vs. item nonresponse

Unit nonresponse means a sampled person provides no data at all. They ignore the invitation, refuse, or can't be reached. Item nonresponse means a person takes part but skips particular questions. Questions on income, health, and other sensitive topics tend to be skipped more often than neutral ones.

Both can bias results. Item nonresponse is usually handled as a missing data problem, with methods such as multiple imputation. Unit nonresponse is what most researchers mean by nonresponse bias, and it's the focus of this guide.

Noncontact vs. refusal

People fail to respond for different reasons, and the reasons matter. Noncontact happens when you never reach someone, such as a person who's rarely home or never sees your email. Refusal happens when someone is reached and declines. Each type is linked to different traits, so each can bias different variables.

When participants drop out of a longitudinal study after taking part, that's attrition. It works the same way, and our guide to selection bias covers it in depth.

Why Doesn't a High Response Rate Guarantee Unbiased Results?

Many researchers treat the response rate as a quality score, and methods textbooks often give minimum thresholds. The evidence doesn't support using any fixed number as a test of bias. To see why, look at what nonresponse bias is made of.

The nonresponse bias formula in plain words

For a simple average, the bias equals the nonresponse rate multiplied by the gap between respondents and nonrespondents.

Bias = nonresponse rate x (respondent mean - nonrespondent mean)

Both parts matter. Suppose 60% of your sample didn't respond, but the two groups differ by only 2 points. The bias is 0.6 times 2, or 1.2 points. Now suppose only 20% didn't respond, but the two groups differ by 10 points. The bias is 0.2 times 10, or 2 points. The survey with the better response rate has the larger bias.

Survey methodologists also describe this in terms of response propensity, each person's likelihood of responding. Bias appears when that likelihood is correlated with the variable you measure. If the people most likely to answer also score high on your outcome, your estimate will run high.

What the research shows

Robert Groves, then director of the University of Michigan Survey Research Center, reviewed 30 studies that measured nonresponse bias directly. Together they produced 235 separate estimates. He found that nonresponse bias did occur, sometimes at large levels. But the nonresponse rate explained only about 11% of the variation in bias across those estimates.

Most of the variation was within surveys, between different questions, rather than between surveys with different response rates. Groves concluded that there's no minimum response rate below which estimates are necessarily biased. He also showed that a higher response rate reduces the risk of bias, not the bias itself (Groves, 2006).

When raising the response rate makes bias worse

Efforts to raise response rates don't affect every group equally. If they mainly bring in people who resemble those already responding, the gap with the remaining nonrespondents can grow. Groves describes an exit poll experiment in which a pen incentive raised the overall response rate. But it raised responses more among one party's voters than the other's. The higher-response condition ended up with more bias in its vote estimates.

The lesson isn't to stop chasing responses. It's to know which groups your efforts reach. Targeted follow-up with the groups least likely to respond does more for bias than a blanket push for a higher rate.

So treat any response rate threshold as a rough signal of risk, not a pass or fail test. A 35% response rate with a documented nonresponse analysis is often more convincing than a 70% rate with no analysis. Some journals and fields still expect particular response rates, so check your target journal's guidelines too.

Nonresponse Bias Examples

These examples show how the reason for nonresponse connects to the outcome. In each case, the people who are hardest to reach differ on the very thing being measured.

Time-use surveys

A survey estimates how much time people spend at home. Interviewers call households, and people who are rarely home are the hardest to reach. Their absence makes the estimate of time spent at home too high. Here, the variable being measured is itself the cause of nonresponse.

Household wealth surveys

Surveys of household finances often struggle to reach the wealthiest households. In one study Groves reviewed, tax records showed that respondents had far less wealth than the full sample. As a result, estimates of average wealth and income came out too low.

A graduate student survey on food insecurity

A doctoral student at Ohio State emails a survey on food insecurity to all graduate students. Students working extra jobs to cover expenses have the least time to respond. They're also the most likely to be food insecure. The survey will probably underestimate food insecurity, even with a respectable response rate.

Skipped income questions

A survey on household financial risk tolerance asks for annual income. Respondents at both ends of the income range may skip the question more often than those in the middle. Dropping cases with missing income would then distort any analysis that uses income as a predictor.

How to Detect Nonresponse Bias

You can't observe the answers nonrespondents would have given. But you can gather indirect evidence about whether they differ from respondents. Plan these checks before data collection, because several depend on records you must keep as you go.

1. Compare respondents with your sampling frame. If your frame includes details on everyone sampled, compare respondents and nonrespondents on those details. Employee rosters, student records, and membership lists often include age, department, or tenure.
2. Compare with external benchmarks. Check your respondents against census figures or a high-quality national survey. Large gaps on demographic variables are a warning sign for your outcome variables too.
3. Run a wave analysis. Record how many reminders each person needed before responding. Then compare early, middle, and late responders on your key variables.
4. Follow up a subsample of nonrespondents. Make an intensive effort to reach a random subset, perhaps with a shorter survey or a different mode. Their answers give you direct evidence about the people you missed.
5. Compare alternative adjustments. Estimate your key results with and without weighting, using different weighting variables. If the conclusions hold across approaches, they're more credible.

How wave analysis works in practice

Wave analysis rests on the idea of a continuum of resistance. People who respond only after several reminders are assumed to resemble those who never respond. If late responders look like early responders, nonresponse is less likely to have shifted your results.

A statewide study of SARS-CoV-2 in Indiana shows the method at work. In April 2020, researchers invited a random sample of residents drawn from state tax records. Only 23.6% took part, a rate many reviewers would question.

The team grouped the 3,658 participants by the number of reminders they'd needed: none, one, or two. Across the three waves, they found no significant differences in demographics, health status, symptoms, or infection rates (Duszynski et al., 2022). That pattern suggests the low response rate didn't distort the main prevalence estimate.

The method has a limit. Late responders aren't always like nonrespondents, and some studies have found the two groups differ sharply. Use wave analysis as one piece of evidence, alongside a frame or benchmark comparison where you can.

What Are Some Solutions to Nonresponse?

The best solutions work at the design stage, before any data are collected. The aim is to reach the groups least likely to respond, not just to raise the overall rate.

• Build a frame with auxiliary data. Choose a sampling frame that records useful details on everyone. Those details let you check and adjust for nonresponse later.
• Plan several contacts. Send an advance notice, the invitation, and two or three reminders. Each contact reaches some people the previous one missed.
• Offer more than one mode. Pair an online survey with paper or phone options. Our guide to methods of collecting data compares the main modes.
• Keep the burden low. Shorter questionnaires lose fewer people. Cut every question you don't need for your analysis.
• Explain the purpose and sponsor. People respond more when they know why a study matters and who runs it. Academic and government sponsors tend to get higher response than commercial ones.
• Use incentives with care. Incentives raise response rates, but they don't appeal to every group equally. Check whether they draw in the groups you're missing.
• Focus effort where response is lowest. Track response by subgroup as data arrive. Then direct follow-up toward the groups falling behind.

Plan your sample size for expected nonresponse too. Our guide on how to calculate sample size shows how to build that into your target. A probability design also helps, and our guide to probability sampling explains how random selection protects the step from frame to sample.

Can Weighting Correct Nonresponse Bias?

Weighting is the most common correction after data collection. It gives more weight to respondents from underrepresented groups, so the sample matches known population totals. Common methods include weighting class adjustments, raking, and propensity score weighting.

Weighting works only under a strong assumption. Within each weighting group, respondents must resemble nonrespondents on your outcome. If young men who responded differ from young men who didn't, weighting by age and sex won't remove the bias.

That assumption usually can't be tested with the data you have, and Groves notes that some adjustments can make estimates worse. Treat weighting as a sensitivity check, and report your results with and without it. Our guide to population vs. sample covers the population figures that weighting relies on.

How to Report Nonresponse Bias in Your Methods Section

Reviewers want to know three things: how many people you lost, whether you checked who they were, and what that means for your conclusions. A strong report covers each in a few sentences.

1. Report the response rate and how you calculated it. State the number invited, the number eligible, and the number who completed the study. Name the formula if you used a standard one, such as an AAPOR response rate.
2. Describe the checks you ran. Say which comparisons you made and what they showed. Include the numbers, not just the conclusion.
3. Report any weighting. Name the weighting variables and say whether results changed when you applied the weights.
4. State the likely direction of remaining bias. Say which estimates are most at risk, and whether they're probably too high or too low.
5. Match your claims to your evidence. If you couldn't check for nonresponse bias, say so plainly and narrow your claims.

The strong version isn't much longer, but it tells reviewers how much weight to give the estimate. Our guide to writing a methods section reviewers won't reject covers the rest of the section.

Common Mistakes About Nonresponse Bias

• Treating the response rate as proof. A high rate lowers the risk but doesn't rule out bias. A low rate raises the risk but doesn't prove it.
• Confusing it with response bias. Nonresponse bias concerns who is missing. Response bias concerns whether the answers you got are accurate.
• Assuming a bigger sample fixes it. More invitations bring in more of the same kind of respondent. The bias stays, measured more precisely.
• Checking only demographics. Matching the population on age and sex doesn't show that you match it on your outcome.
• Weighting without saying so. Readers need to know which weights you used and whether they changed your findings.
• Planning the analysis too late. Wave analysis needs reminder records, and frame comparisons need frame data. Both are hard to recover afterward.

Professional Editing for Your Methods and Limitations Sections

A careful nonresponse analysis only helps if reviewers can follow it. Editor World provides journal article editing, dissertation editing, and academic editing. Every editor is a native English speaker from the United States, the United Kingdom, or Canada. Every document is edited by a person, never by AI. You can choose an editor with experience in your field, and the first 300 words of any document are edited at no cost, no commitment.

Frequently Asked Questions About Nonresponse Bias

Related Guides

For the full map of bias categories, see our research bias guide. It shows where nonresponse fits among the five main types.

Our guide to selection bias covers self-selection, attrition, and survivorship bias, the other ways a sample can drift from its population.

To understand distortion in the answers people give, read our guide to response bias, including social desirability and acquiescence.

For designs without random selection, our guide to non-probability sampling explains what each approach costs in representativeness.

The research methodology guide shows how sampling and bias decisions fit into the overall structure of a study.

Reviewed by the Editor World editorial team under the direction of Patti Fisher, PhD, founder. Editor World has provided 100% human editing by native English editors since 2010. About Editor World