Margin of Error: Formula and Interpreting

What is the Margin of Error?

The margin of error (MOE) for a survey tells you how near you can expect the survey results to be to the correct population value. For examplea survey indicates that 72% of respondents favor Brand A over Brand B with a 3% margin of error. In this casethe actual population percentage that prefers Brand A likely falls within the range of 72% ± 3%or 69 – 75%.

The margin of error calculates a distance from the survey’s value in which the actual population value is likely to occur. It assesses the precision of a survey’s estimates. A smaller margin of error suggests that the survey’s results will tend to be close to the correct values. Converselylarger MOEs indicate that the survey’s estimates can be further away from the population values.

Surveys frequently use random samples to estimate population percentages. These population valuesalso known as parametersare almost always unknown because it is infeasible to measure entire populations. Insteadsurveys use samples to estimate them. Unfortunatelythese estimates seldom equal the correct value exactly due to random sampling error. The MOE is an inferential statistics procedure related to confidence intervals that evaluates the uncertainty involved when using random samples to estimate a population’s characteristics.

In this postlearn about the margin of error formula and how to calculate and interpret them.

Related posts: Populations and SamplesParameters vs. Statisticsand PopulationsParametersand Samples in Inferential Statistics

Confidence Levels

Like confidence intervalsthe margin of error has a confidence level. Different random samples drawn from the same population are likely to produce slightly different estimates. If you draw many random samples and calculate a margin of error around each estimatea percentage of those ranges will contain the population value.

Confidence levels are the long-run percentage of times that the population value falls within the MOE. Standard confidence levels are 9095and 99%. 95% is the most common. For surveys that use a 95% confidence levelyou’d expect 95% of their results to have margins of error that contain the population value.

For the image belowimagine that you draw 20 random samples and administer the same survey to all of them. Then you calculate the margin of error for an item on all 20 surveys. If you use a 95% confidence levelyou’d expect that 19 out of the 20 intervals will contain the correct population value.

Related posts: Confidence Intervals and Sample Estimates Are Always Wrong (to Some Extent)!

Maximum Margin of Error for a Survey

In a surveythe size of the margin of error varies depending on the percentage. Surveys frequently cite its maximum MOE. That’s the value you’ll see in the media when they report a survey’s margin of error. Knowing this value is beneficial because the margins of error for all percentages in a survey will not exceed this value.

The largest margin of error for a given sample size occurs when the percentage is 50%. For example50% of the respondents prefer Candidate Smith. As the percentage for an item moves away from 50%either higher or lowerthe MOE decreases.

Let’s see this phenomenon in action graphically! The graph below displays the 95% margin of error for a sample size of 1,000. The vertical axis represents the MOEwhile the horizontal axis indicates percentages in the survey. The highest margin of error occurs at the red line where the survey result is 50%and it diminishes in both directions from that peak.

In the margin of error formula sectionI’ll show you how to find the maximum and other MOEs.

Download the Excel file I made to create this graph: Margin Of Error. In this fileyou can change the sample size to see how it affects the MOE for all percentages.

Margin of Error Formula

Surveys frequently use proportions and percentages in their results. For example92% agree with a particular decision. Consequentlythe margin of error formula for surveys relates to percentages. When you add and subtract the MOE from the survey proportionyou obtain a confidence interval of the proportion.

In statisticsmost margin of error/confidence interval calculations involve multiplying a critical value by a standard error. In the following formulathe square root is the standard error of the proportionand you multiply it by a critical Z-value. Learn more about standard errors.

The margin of error formula is the following:

Where:

• The Z-value is the critical Z-value that corresponds to your confidence level.
• p is the sample proportion or percentage.
• n is the sample size.

You set the confidence level by choosing the critical Z-value. Below are the Z-values for the standard confidence levels.

Confidence Level	Z-value
90%	1.64
95%	1.96
99%	2.58

Related posts: Critical ValuesZ-scoresand Z-table

Finding the Maximum Margin of Error Formula

To find the maximum MOE that surveys typically reportwe need to set p to 0.5 and use the Z-value for our confidence level. 95% is the standard confidence level. ConsequentlyI’ll use the corresponding Z-value of 1.96. Because we’re setting the values of these two inputs (Z and p)the maximum margin of error formula becomes a direct transformation of the survey’s sample sizeas shown below.

For any surveyall we need to enter into the formula is the sample size to find its maximum margin of error. Each sample size has its own maximum value. For exampleif our sample size is 1000our maximum MOE is the following:

This maximum MOE corresponds to the peak in the chart above.

Finding the Margin of Error for Other Percentages

It’s easy to use the margin of error formula for other percentages. Simply enter the percentage as a proportion (i.e.75% = 0.75)sample sizeand Z-score into the formula.

For examplea survey item finds that 75% of the respondents agree with the governor’s policy. 1000 individuals responded to this survey. To find the 95% margin of error for this percentagewe do the following:

We can be 95% confident that the population percentage for this survey item is within 75% ± 2.684%or 72.316 – 77.684%.

Additionallynotice that the MOE for this specific percentage (75%) is 2.684% versus the maximum MOE of 3.099% for 50% when you have a sample size of 1,000. You can find both of these points on the earlier chart.

Caveats and Cautions for MOEs

The margin of error only accounts for random sampling error. It cannot factor in wording biasesnon-representative samplesdifficulties that exclude particular types of respondentsetc.

Additionallythese MOEs are for individual percentages and not the differences between responses. For examplesuppose 51% of respondents prefer Candidate A and 48% prefer Candidate B. If the margin of error is 4%the media will often state that it’s a statistical dead heat or tie because the difference between the two candidates (3%) is smaller than the margin of error (4%). Learn more about the Challenges of Political Polling.

While that is trueit’s an overly simplistic method because it doesn’t indicate how large the difference between two percentages must be to be statistically significant. To evaluate the differences between two percentagesyou need to calculate a different type of interval—the confidence interval of the difference between two proportions. I use that type of test in my post about the effectiveness of flu vaccines.

Howeverwhen you only know the reported margin of error for individual percentagesyou might still be able to draw some conclusions. If the difference between the two percentages is twice the MOEit is statistically significant. Additionallydifferences slightly smaller than 2 * MOE can also be significantbut you’d need to perform the correct test to know for sure.

I write about a similar phenomenon when comparing group means using confidence intervals of the differences rather than the individual means.