ActiVote Polling Methodology
ActiVote is a free app that helps any US voter confidently vote in every election. The app is free, non-partisan and we don’t share anyone’s personal data with anyone else.
Every day, thousands of our users interact with the app to become better informed about our democracy. They answer survey questions to get a position in the Political Matrix, rate their representatives to show approval or disapproval, vote on bills to see how they overlap with their legislators, and rank candidates for the races they can vote in to prepare for when they will fill in their official ballot.
In the app, our users can see in real-time what our users collectively think about a particular topic. And regularly, we highlight a survey question, bill, or legislator in a blog post where we show a poll on that topic, weighted for the composition of the electorate.
The data for our election polls are the collective rankings from our users for races they can vote in. Just like any other poll, the individual answers of users are private, but the collection of the preferences of all users taken together can provide insight in the status of the race.
Which Races We Poll
For the 2023 and 2024 election season ActiVote will publish any state-wide race (primaries, run-offs, general elections and national polls for president and the house of representatives) where we have at least 200 participants in a poll. We don’t cherry pick races in any way. We simply publish every single poll for which we have enough rankings. For all other races, unweighted polls can be found directly in the app.
Timing of our Polls
Unlike virtually any other pollster, ActiVote does not actively sollicit replies from our users. Instead, whenever a user is ready to make a choice in a particular race, they may elect to rank the candidates for that race. From that moment on, they are potentially included in the poll for that race. Any user can change their mind at any point in time, and change their ranking if they so desire.
Whenever we collect the data for a poll, we take the most recent rankings that have been set during the past days/weeks, such that we have sufficient rankings to get to a reasonable sample size. In the past (especially polls in 2022 and 2023), we would specify the period during which rankings could have been initially created. However, that did not give an accurate picture of when most of the users had created or especially last updated their rankings. Therefore, since February 2024, we specify the Median Field Date, which is more in line with other pollsters and polling aggregators.
How to interpret the data
We correct our polls for underrepresentation of any group, based on age, gender, party affiliation etc., because our users are not a carefully, randomly selected group of people. The result is a poll that better represents the full electorate.
Of course, there are limits to how accurate any poll can be depending on the number of people who participate. For the polls based on the smallest number of people that we publish (200 participants), even perfect polls would on average still be 2.7% off. In practice, however, polls are not perfect, and the average error will therefore be larger.
After the election returns have come in for a race in which we published a poll, we check how we did. Based on these evaluations we determine the expected average error that ActiVote’s polls generally have. For each future poll, we include that expected average error as an indicator for the reader to help interpret the results.
Thus, if we indicate for a particular poll that the expected average error is 5%, it means that in past polls of this sample size, ActiVote has been off by on average 5%. While some have been less than 5% off, others will have been off by more than 5%. On average, though, these kinds of polls will be about 5% off.
Why we publish these polls
Non-probabilistic polls with small samples sizes will have larger deviations than the average poll that gets significant attention by the national media and polling aggregators. Thus, if other polls are available for these races, please check out those other sources as well. However, many races are not polled at all by traditional pollsters, because they are not deemed interesting enough for anybody to fund the poll.
That is where ActiVote comes in: our users are ranking candidates in over 10,000 races nation-wide and less than 1% of those races get attention from other pollsters. Providing at least some insight into those other races, will be useful to our users.
With growing usage of ActiVote, we will be able to publish more and more polls, with increasing sample sizes and improved accuracy, all with the goal to help more people confidently vote in every election.
AAPOR’s Transparency Initiative
The American Association of Public Opinion Research (AAPOR) has created the Transparency Initiative, which aims to promote methodological disclosure by survey organizations. ActiVote has applied to and has been accepted into the Transparency Initiative
You can find ActiVote’s disclosures as mandated by the Transparency Initiative here.