ActiVote Polling Methodology
ActiVote is a free app that helps any US voter confidently vote in every election. The app is free, non-partisan and we don’t share anyone’s personal data with anyone else.
Every day, thousands of our users interact with the app to become better informed about our democracy. They answer survey questions to get a position in the Political Matrix, rate their representatives to show approval or disapproval, vote on bills to see how they overlap with their legislators, and rank candidates for the races they can vote in to prepare for when they will fill in their official ballot.
In the app, our users can see in real-time what our users collectively think about a particular topic. And regularly, we highlight a survey question, bill, or legislator in a blog post where we show a poll on that topic, weighted for the composition of the electorate.
The data for our election polls are the collective rankings from our users for races they can vote in. Just like any other poll, the individual answers of users are private, but the collection of the preferences of all users taken together can provide insight in the status of the race.
Which Races We Poll
For the 2022 election season ActiVote publishes polls about any state-wide race (primaries, run-offs, and general elections) where we have at least 50 participants in a poll. We don’t cherry pick races where we are most confident. We simply publish every single poll for which we have enough rankings. We regularly update polls if sufficient new rankings have come in, up to the weekend before the election day for that race.
Over time we plan to expand our polls to any race that has at least 50 participants in our polls and not just state-wide races. For now, you can already find the unweighted poll for those races directly in the app.
How to interpret the data
We correct our polls for underrepresentation of any group, based on age, gender, party affiliation etc., because our users are not a carefully, randomly selected group of people. The result is a poll that better represents the full electorate.
Of course, there are limits to how accurate any poll can be depending on the number of people who participate. For the polls based on the smallest number of people that we publish (50 participants), even perfect polls would on average still be 5.6% off. In practice, however, polls are not perfect, and the average error will therefore be larger.
After the election returns have come in for a race in which we published a poll, we check how we did. Based on these evaluations we determine the expected average error that ActiVote’s polls generally have. For each future poll, we include that expected average error as an indicator for the reader to help interpret the results.
Thus, if we indicate for a particular poll that the expected average error is 10%, it means that in past polls of this sample size, ActiVote has been off by on average 10%. While some have been less than 10% off, others will have been off by more than 10%. On average, though, these kinds of polls will be about 10% off.
Why we publish these polls
Non-probabilistic polls with small samples sizes will have larger deviations than the average poll that gets significant attention by the national media and polling aggregators. Thus, if other polls are available for these races, please check out those other sources as well. However, many races are not polled at all by traditional pollsters, because they are not deemed interesting enough for anybody to fund the poll.
That is where ActiVote comes in: our users are ranking candidates in over 10,000 races nation-wide and less than 1% of those races get attention from other pollsters. Providing at least some insight into those other races, will be useful to our users.
With growing usage of ActiVote, we will be able to publish more and more polls, with increasing sample sizes and improved accuracy, all with the goal to help more people confidently vote in every election.
AAPOR’s Transparency Initiative
The American Association of Public Opinion Research (AAPOR) has created the Transparency Initiative, which aims to promote methodological disclosure by survey organizations. ActiVote has applied to and has been accepted into the Transparency Initiative
You can find ActiVote’s disclosures as mandated by the Transparency Initiative here.