Learn more about analytics and research best practices, as well as real world examples and solutions for nonprofits.

RECENT POSTS

Nov 4, 2020

Fake Polls

Polling is getting a lot of attention today. Bad attention.

This is the second election cycle in a row where the polls were pointing towards a solid win for the Democratic Presidential candidate, but the actual vote went the other way.

We call the issue is called Sampling Bias: meaning, the people who participated in the survey don’t accurately reflect the population the poll is supposed to be studying.

Now, the (former) Gold Standard of polling forecasts, Nate Silver’s 538, will tell you that all the results are within the poll’s margin of error. And while the results were super close, I don’t agree with him.

Take a look at this PDF of 538’s final forecast compared with the current results (as of the afternoon of November 4th).If it were merely sampling error, we would see the error go both ways. It is not doing that. The error is consistently under projecting support for President Trump. Sometimes by a lot.

538 pdf

This means that the polls have a serious methodological problem.

As a survey researcher, this is deeply troubling – it means whatever weighting “fixes” pollsters like 538 applied to mitigate the 2016 errors simply did not work. It is also troubling because it further erodes confidence in any survey research findings. Now, whatever the result end up being in this exceptionally tight election, the real loser is polling sciences.

Mentors: Phil Stolberg

As we finish one year and head into the next, I thought it would be appropriate to recognize some of the key mentors I have had that have shaped my career. So, this 4-part series focuses on those who have helped me along. A big thank you to each and every one of them....

read more

Mentors: Steven Screen

As we finish one year and head into the next, I thought it would be appropriate to recognize some of the key mentors I have had that have shaped my career. So, this 4-part series focuses on those who have helped me along. A big thank you to each and every one of them....

read more

Mentors: Jon Van Wyk

As we finish one year and head into the next, I thought it would be appropriate to recognize some of the key mentors I have had that have shaped my career. So, this 4-part series focuses on those who have helped me along. A big thank you to each and every one of them....

read more

ARCHIVES

© 2022 Analytical Ones