(This article was originally published at » R, and syndicated at StatsBlogs.)
— layout: page subheadline: “Discussion” title: “Understanding Margins of Error for Small Populations” teaser: “A comment on why does a pollster reported a narrow margin of error despite the small sample.” date: 2015-10-13 09:00:00 categories: - R tags: - R - Survey - Margin of error comments: true show_meta: false — I receive an email asking for a possibly wrong doing with the margin of error calculated by the Datafolha’s newest poll. This polling firm often survey the population with a regular sample (1000/2400), usually calculating a margin of error of +/-3% to +/-2%. However, this time, the pollster sampled 340 congressmen out of 594 (513 deputies + 81 senators), reporting a margin of error of 3%. This puzzled the attentive reader who knows the margin of error is an inverse square root function of the sample size, so she asked: “Is the margin of error reported here is correct? Shouldn’t it be something around 5.4% given the reduced sample size?
First of, I found the poll very interesting. Datafolha should conduct polls like that more often. So, we can get to know about the average position of our representatives.
My answer to the inquiry was “sort of”. Firstly, given the fact the population of congressmen is a finite population, we must apply a corrector factor for finite populations (FPC), which will reduce the standard error of the mean. This corrects for the fact that although the sample is small, it comes from a small population too. Below, I go through the calculation steps, which will show that by omitting decimal digits, the pollster arbitrarily narrowed the confidence interval (2 * the margin of error) in about 1%. It’s not that usual to round margins of error when reporting them, as a decimal digit may imply in huge differences in the sample size as illustrated in the graph from Marcelino and Angel’s paper.
For pedagogical reasons, I won’t go through the simplified formula. So, let do it step-by-step.
The derivation of the maximum margin of error formula is given by:
MOE = (1.96)sqrt[p(1-p)/n] = (1.96)sqrt[(0.5)(0.5)/n] = (1.96)sqrt[(0.25)/n] = (1.96)(0.5)sqrt[1/n] = (0.98)sqrt(1/n)
If we sample a n = 340, we get MOE = (0.98)sqrt(0.05423) = 0.05315, or 5.315%.
Finite Population Margin of Error
The two formulas above are accurate if the random samples are drawn from extremely large populations. However, when the total population for a survey is much smaller, or the sample size is more than 5% of the total population, we should multiply the margin of error by the Finite Population Correction Factor (FPCF). The forumula is
FPCF = sqrt[(N-n)/(N-1)]
If we multiply the resulting MOE above by the FPCF, we get:
MOE with FPCF = sqrt[(594-340)/(594-1)](0.05315) = sqrt[254/594](0.05315) = (0.8)(0.05315) = 0.0425
So these survey results have a maximum margin of error of +/- 4.25%.
If 67% of the respondents gave the same answer, then the margin of error would be
Os contrastes mais significativos de posição aparecem quanto à criminalidade: 74% dos congressistas acreditam que a principal causa do problema seja “a falta de oportunidades iguais para todos”.
sqrt[(594-340)/(594-1)](1.96)sqrt[(0.67)(0.33)/865] = sqrt[1536/2400](1.96)sqrt[0.000255607] = (0.8)(1.96)(0.0159877) = 0.025069, or 2.5069%.
Basic Statistical Testing for Margin of Error Between Two Proportions/Percentages Step 1: Desired confidence level: 99% 95% 90% Step 2: Proportion: % Step 3: Sample size: Step 4: Population size: infinite actual/estimated use ‘infinite’ when the group is larger than 1, or you don’t know the actual size of the overall population
Margin of error: ? %
see formula
Margin of Error Between Two Proportions
Formula
Large (Infinite) Populations
e = zs/sqrt(n)
where:
z = 2.576 for 99% level of confidence 1.96 for 95% level of confidence 1.645 for 90% level of confidence
s = sqrt(p(1-p)) p = proportion
n = sample size
Finite Populations
e = zs/sqrt(nadj)
where:
z = 2.576 for 99% level of confidence 1.96 for 95% level of confidence 1.645 for 90% level of confidence
s = sqrt(p(1-p)) p = proportion
nadj = (N-1)n/(N-n) n = sample size N = population size
Basic Statistical Testing for Margin of Error Between Two Means Step 1: Desired confidence level: 99% 95% 90% Step 2: Standard deviation: Step 3: Sample size: Step 4: Population size: infinite actual/estimated: use ‘infinite’ when the group is larger than 1, or you don’t know the actual size of the overall population
Margin of error: ±
see formula
Margin of Error Between Two Means
Formula
Large (Infinite) Populations e = zs/sqrt(n)
where:
z = 2.576 for 99% level of confidence 1.96 for 95% level of confidence 1.645 for 90% level of confidence
s = standard deviation
n = sample size
Finite Populations e = zs/sqrt(nadj)
where:
z = 2.576 for 99% level of confidence 1.96 for 95% level of confidence 1.645 for 90% level of confidence
s = standard deviation
nadj = (N-1)n/(N-n) n = sample size N = population size
There is a function in my package called sample.size, which I use for the approximate sample size and margin of errors, that one may use for calculating this right way. The function was planed to return the sample size, but one can play around with desired margin of error and population size to compute those.
Please comment on the article here: » R
