ActiVote Polling Methodology
Our Data
ActiVote is a free app that helps any US voter confidently vote in every election. The app is free, non-partisan and we don’t share anyone’s personal data with anyone else.
Every day, thousands of our users interact with the app to become better informed about our democracy. They answer survey questions to get a position in the Political Matrix, rate their representatives to show approval or disapproval, vote on bills to see how they overlap with their legislators, and rank candidates for the races they can vote in to prepare for when they will fill in their official ballot.
In the app, our users can see in real-time what our users collectively think about a particular topic. And regularly, we highlight a survey question, bill, or legislator in a blog post where we show a poll on that topic, weighted for the composition of the electorate.
The data for our election polls are the collective rankings from our users for races they can vote in. Just like any other poll, the individual answers of users are private, but the collection of the preferences of all users taken together can provide insight in the status of the race.
Which Races We Poll
For the 2022 primary election season ActiVote published polls about any state-wide race (primaries and run-offs) where we had at least 50 participants in a poll. We didn’t cherry pick races in any way. We simply published every single poll for which we had enough rankings.
For the 2022 general election season ActiVote publishes polls once a week for any state-wide race where we have at least 100 participants in a poll who choose to rank their candidates in the last 90 days. We exclude polls where after weighting for all characteristics like age, gender, affiliation, ethnicity, income, education, region and political leanings the sample deviates too much from the state’s target audience. For each poll we include both the target and actual characteristic distribution in the “crosstab” image underneath the poll.
Over time we plan to expand our polls beyond just state-wide races to any race that has at least 50 participants. For now, you can already find the unweighted poll for those races directly in the app.
How to interpret the data
We correct our polls for underrepresentation of any group, based on age, gender, party affiliation etc., because our users are not a carefully, randomly selected group of people. The result is a poll that better represents the full electorate.
Of course, there are limits to how accurate any poll can be depending on the number of people who participate. For the polls based on the smallest number of people that we publish (50 participants), even perfect polls would on average still be 5.6% off. In practice, however, polls are not perfect, and the average error will therefore be larger.
After the election returns have come in for a race in which we published a poll, we check how we did. Based on these evaluations we determine the expected average error that ActiVote’s polls generally have. For each future poll, we include that expected average error as an indicator for the reader to help interpret the results.
Thus, if we indicate for a particular poll that the expected average error is 10%, it means that in past polls of this sample size, ActiVote has been off by on average 10%. While some have been less than 10% off, others will have been off by more than 10%. On average, though, these kinds of polls will be about 10% off.
Why we publish these polls
Non-probabilistic polls with small samples sizes will have larger deviations than the average poll that gets significant attention by the national media and polling aggregators. Thus, if other polls are available for these races, please check out those other sources as well. However, many races are not polled at all by traditional pollsters, because they are not deemed interesting enough for anybody to fund the poll.
That is where ActiVote comes in: our users are ranking candidates in over 10,000 races nation-wide and less than 1% of those races get attention from other pollsters. Providing at least some insight into those other races, will be useful to our users.
With growing usage of ActiVote, we will be able to publish more and more polls, with increasing sample sizes and improved accuracy, all with the goal to help more people confidently vote in every election.
AAPOR’s Transparency Initiative
The American Association of Public Opinion Research (AAPOR) has created the Transparency Initiative, which aims to promote methodological disclosure by survey organizations. ActiVote has applied to and has been accepted into the Transparency Initiative
You can find ActiVote’s disclosures as mandated by the Transparency Initiative here.
