Margin of Error help
Overview
Margin of Error will generate a confidence interval on a percentage, provided you also supply the sample size and the confidence level. For example, if you polled 1000 people and 23.7% answered a question in a given way, Margin of Error will calculate the uncertainty (the margin of error) in that poll.
Margin of Error is free. If you use it for a published paper or talk, please list Margin of Error in your references or mention it in your acknowledgements. In either case, indicate that the program is available at this website or through the Mac App Store. Journals differ in citation format, but this is a sample citation with the primary information most journals will require:
Holland, S.M., 2021. Margin of Error, version 1.1. www.huntmountainsoftware.com
What to enter
Three values are needed to generate a confidence interval for a percentage.
First, the measured or observed percentage is entered. For example, 45% would be entered as 45. Don’t add the percent symbol.
Next, the sample size is entered — that’s the total number of items counted. For example, if 1000 people were polled and exactly 45% (450 people) gave a particular answer, the sample size would be 1000. Sample size is always an integer, a whole number without any decimal.
Last, the statistical confidence level is entered. This is by default set at 95%, a common confidence level. Confidence levels may be higher (99%, 99.9%) or lower (90%, 80%, 50%), depending on a given problem. If you’re unsure of what confidence level to use, check with what other people in your field use for similar problems. Confidence level is entered without the percent symbol. For example, a 95% confidence level is entered as 95.
How the confidence interval is calculated
Confidence intervals on percentages are calculated using a binomial probability distribution. In short, the observed percentage is used for the probability of success and the probability of any given number of successes is calculated using the binomial formula. The area under this probability distribution is summed until the desired level of confidence is reached and this defines the confidence interval. The calculations are tedious by hand, but are quickly done by Margin of Error.
The confidence interval engine is based on source code in Raup (1991), which presents a full description of the rationale behind this approach for calculating confidence intervals on percentages.
What a Confidence Interval Means
Confidence intervals are meant to bracket the actual percentage that would be measured if you could measure the entire population, which is often impractical in terms of time, money, or both.
The confidence level describes how often your confidence interval will bracket the true percentage. For example, if you typically create 95% confidence intervals, then 95 of 100 confidence intervals, on average, will bracket the true percentages.
It is critical to realize that this does not apply to a single example. It is a widespread misconception that there is a 95% chance that the true percentage lies within a confidence interval you have generated. This cannot be the case because the true percentage exists; you may not know its value but there is no chance involved in its value.
Confidence levels become broader as the confidence level increases. In other words, to become more confident that your confidence interval brackets the true percentage, the confidence interval must get wider.
Extended discussions of confidence intervals are available in most statistical textbooks. Wikipedia has a good discussion of the topic as well.
Recommended reading
Raup, D.M., 1991, The future of analytical paleobiology. In N.L. Gilinsky and P.W. Signor, eds., Analytical Paleobiology, Paleontological Society Short Courses in Paleontology No. 4, p. 207–216