ActiVote Polling Methodology
Why we poll
ActiVote is a free app that helps any US voter confidently vote in every election. The app is free, non-partisan and we don’t share anyone’s personal data with anyone else.
Every day, thousands of our users interact with the app to become better informed about our democracy. They answer survey questions to get a position in the Political Matrix, rate their representatives to show approval or disapproval, vote on bills to see how they overlap with their legislators, and rank candidates for the races they can vote in to prepare for when they will fill in their official ballot.
Part of the interactive experience in our app is to get feedback on what other people think. To provide such feedback, we developed ActiVote’s polling engine such that for all survey questions we can show results that are accurately weighted for the composition of the electorate. That same polling engine allows us to weigh election polls for any of the thousands of election races all around the nation that are covered in the app.
For any race where we have a sufficiently large sample size, we publish our findings.
All Data remains anonymous
The data for our election polls are the collective preferences from our users for races they can vote in. Just like any other poll, the individual answers of users are private, but the collection of the preferences of all users taken together can provide insight in the status of the race, without in any way jeopardizing the privacy of individual users.
Which Races We Poll
For the 2024 election season, ActiVote will publish any state-wide races with at least 400 participants and national polls for president with typically 1000+ participants. We also plan to publish predictions for the house of representatives and the electoral college as a whole.
We don’t cherry pick races in any way. We simply publish every single poll for which we have enough preferences. For all other races, unweighted polls can be found directly in the app.
Timing of our Polls
ActiVote does not actively solicit replies from our users. Instead, whenever a user is ready to make a choice in a particular race, they may elect to indicate which candidate(s) they prefer in that race. From that moment on, they are included in the poll for that race. Any user can change their mind at any point in time and their new preference is used instead.
Whenever we collect the data for a poll, we take the most recent preferences that have been set during the past days/weeks, such that we have sufficient preferences to get to the desired sample size. We specify the “field period” (typically a few weeks) including the median field date for each poll.
How to interpret the data
We weight our polls for underrepresentation of any group, based on age, gender, party affiliation, ethnicity, income, education and region because our users are not a carefully, randomly selected group of people. The result is a poll that aims to correctly represent the full electorate.
Of course, there are limits to how accurate any poll can be depending on the number of people who participate. For the polls based on the smallest number of people that we publish (400 participants), even perfect polls would on average still be 4.0% off.
In practice, however, polls are not perfect, and the average error therefore could be larger.
After the election returns have come in for any race for which we have a poll (independent of whether we had a large enough sample to publish the poll or not) we check how we did. Based on these evaluations we determine the expected average error that ActiVote’s polls generally have. For each future poll, we include that expected average error as an indicator for the reader to help interpret the results.
Thus, if we indicate for a particular poll that the expected average error is 5%, it means that in past polls of the same sample size, ActiVote has been off by on average 5%. See here For more details on ActiVote’s polling accuracy, see.
538 Transparency Questions
538 has published pollster ratings for many years to help people assess the quality of the polls of a particular pollster. 538 determines its ratings based on two main criteria:
(1) Accuracy. This component mainly consists of the average error in the polls of a pollster and any possible bias towards either Democrats or Republicans.
(2) Transparency. This component consists of 10 key questions 538 would like to see answered by pollsters about their polls.
ActiVote aims to provide the full transparency 538 seeks. You can find ActiVote’s answers to 538’s transparency questions here.
AAPOR’s Transparency Initiative
The American Association of Public Opinion Research (AAPOR) has created the Transparency Initiative, which aims to promote methodological disclosure by survey organizations. ActiVote is a member of AAPOR’s Transparency Initiative.
You can find ActiVote’s disclosures as mandated by the Transparency Initiative here.
