Falling into the Margin of Error
I'm tired of the news media reporting polls and drawing conclusions that are invalid.
When a poll shows the presidential race as 50%/46% in favor of Bush with a +/- 3% margin of error, the President is not ahead in the poll.
When a subsequent poll shows the race as 49%/47% in favor of Bush with the same margin of error, John Kerry is not gaining ground.
These two polls show the same result: The candidates are statistically tied.
This is why three polls can show three results, one showing the President way ahead, and the other two showing it much closer. If all of them are within the margin of error of each other then they're showing the same result.
Remember that statistics are estimates of an actual, but unmeasurable, value. It would be impossible to poll every voter and report the results in a timely and affordable fashion. So we sample 1000 or 3000 voters and tell you what they think. Based on the sample size and assuming the voters sampled are truly representative of the total universe of voters, statisticians determine a confidence level and margin of error.
A result like 49.681% looks very precise. But if it's the result of a poll that has a 95% confidence and a 2% margin of error, then what it means is "if we could actually poll every single voter, we're 95% sure that the result is would be between 47.681% and 51.681%." That's a pretty big margin when the race is close.
As of today, one poll reports the race at 49.0% to 45.4% in favor of Bush. Last week, this poll reported 47.2% to 46.5%. This poll has a 2% margin of error. While Bush supporters may look at this and see their candidate pulling ahead, the fact is that both of these polls show the same result.
I saw this working in real life last year when working against a local referendum to allow riverboat gambling in Cedar Rapids. We polled 30,000 highly likely voters to find out where they stood and to establish our base on election day. Early results were very favorable for our anti-gambling cause. It looked like we'd have no problem defeating the referendum.
As we talked to more people, however, the results started trending closer (and unbeknownst to us at the time, trending toward the actual outcome). When I calculated the margin of error on the early results, I found that the later results were still within the margin of error. And in the end, the final result of the vote was within the margin of error of the earliest polls (even though the final result was closer than our early polls).
So... always look at the margin of error, and don't worry about trends one way or another until they break through that margin of error.
When a poll shows the presidential race as 50%/46% in favor of Bush with a +/- 3% margin of error, the President is not ahead in the poll.
When a subsequent poll shows the race as 49%/47% in favor of Bush with the same margin of error, John Kerry is not gaining ground.
These two polls show the same result: The candidates are statistically tied.
This is why three polls can show three results, one showing the President way ahead, and the other two showing it much closer. If all of them are within the margin of error of each other then they're showing the same result.
Remember that statistics are estimates of an actual, but unmeasurable, value. It would be impossible to poll every voter and report the results in a timely and affordable fashion. So we sample 1000 or 3000 voters and tell you what they think. Based on the sample size and assuming the voters sampled are truly representative of the total universe of voters, statisticians determine a confidence level and margin of error.
A result like 49.681% looks very precise. But if it's the result of a poll that has a 95% confidence and a 2% margin of error, then what it means is "if we could actually poll every single voter, we're 95% sure that the result is would be between 47.681% and 51.681%." That's a pretty big margin when the race is close.
As of today, one poll reports the race at 49.0% to 45.4% in favor of Bush. Last week, this poll reported 47.2% to 46.5%. This poll has a 2% margin of error. While Bush supporters may look at this and see their candidate pulling ahead, the fact is that both of these polls show the same result.
I saw this working in real life last year when working against a local referendum to allow riverboat gambling in Cedar Rapids. We polled 30,000 highly likely voters to find out where they stood and to establish our base on election day. Early results were very favorable for our anti-gambling cause. It looked like we'd have no problem defeating the referendum.
As we talked to more people, however, the results started trending closer (and unbeknownst to us at the time, trending toward the actual outcome). When I calculated the margin of error on the early results, I found that the later results were still within the margin of error. And in the end, the final result of the vote was within the margin of error of the earliest polls (even though the final result was closer than our early polls).
So... always look at the margin of error, and don't worry about trends one way or another until they break through that margin of error.
posted by Craig Rairdin at 11:01
<< Home