AAPOR Transparency Initiative Disclosures
ActiVote adheres to Disclosure Standards as part of the Transparency Initiative of the American Association of Public Opinion Research (AAPOR), by disclosing important information about the ActiVote surveys. This text provides all relevant disclosures that hold true for every survey executed by ActiVote, while the individual survey reports each contain relevant disclosures specific to that particular survey.
1. Data Collection Strategy
ActiVote polls can be classified as based on an “app panel”. All data is collected automatically when ActiVote users perform a “polling action” in the ActiVote app (iOS, Android, or web). A “polling action” can be selecting a candidate as preferred candidate in an election race, answering a policy question, rating a representative or voting on a bill.
2. Who Sponsored the Research and Who Conducted It?
Every poll is executed by ActiVote Inc., a non-partisan, registered Delaware Public Benefit Corporation and B Corp focused on helping people confidently vote in every election. Individual polls require no specific funding as all data is proactively provided by our users. The only costs involved, the operation of the ActiVote platform, is entirely funded by ActiVote.
3. Measurement Tools/Instruments
Every polling action is available directly in the free ActiVote app. The election race, question, bill, and representative screens have an interactive option to select a candidate, answer a question, vote on a bill, or rate a representative. All screens are available in the app and remain accessible to anyone who wishes to analyze the exact wording and presentation of each survey.
For election polls, users can navigate to two different election screens. Here we show example screens relevant for the 2024 general presidential election, as of early July 2024.
The next three screenshots show the “ranking screen”.
The screen starts with the name of the race (President of the US), then the selected preference(s), followed by the list of candidates in alphabetical order, with profile picture, colored by their party color.
By selecting the “+” in front of a candidate tile, the selected candidate moves to the top of the screen as preferred candidate.
• The left screen shows the screen before any selection has been made.
• The middle screen shows the screen after a user selected Joe Biden as preferred candidate.
• The screen on the right shows the screen after a user selected Donald Trump as their candidate.
The next two screenshots show the “ballot screen”.
Users can select the “ballot screen”, where candidates for all races are shown in a ballot-like format. The screenshot on the left shows the presidential ballot before any choice has been made, while on the right the ballot is shown after Robert Kennedy has been selected.
In summary, for any election, candidates are shown alphabetically (on first name) with their profile pictures, and after a selection has been made, the selected candidate is clearly visible.
4. Population Under Study
Every survey focuses on U.S. voters. For election polls it is further narrowed to likely voters in that specific type of election: presidential, presidential primary, midterm, primary, local, or special. We define likely voters on a continuous scale based on a combination of their personal voting history and the voting history of people with similar characteristics. Thus, a voter with a 90% chance to vote in a particular election will count as 0.9 of a voter, while a voter with a 30% chance to vote will count as 0.3 of a voter.
5. Method Used to Generate and Recruit the Sample
The survey participants are a non-probability sample: anyone who has downloaded our app and starts participating in our polls is included.
Users can identify themselves in any of three ways:
(1) By state, limiting their interaction to state-wide elections,
(2) Full address, allowing interaction on any elections,
(3) Name and full address, allowing us to link them to their voter registration as provided by the voter file from L2.
The level of identification determines how many characteristics we will have available of the user.
We do not use quotas for any characteristic. However, we do have decent representation for all characteristics, with the largest discrepancies being that our users skew a bit younger and more male than the general population.
Users are excluded from our polls if and only if we believe that they are not genuine in their participation in the app. We cannot be transparent about the various checks we perform, as that would render these safeguards less effective, but the goal is to ensure that we include all genuine users and exclude any pretenders.
There are no specific strategies to gain cooperation. However, the app selects for every user a daily action as a suggested action that they may wish to complete, which can be any type of survey question (policy question, voting on a bill, rating a representative, selecting a candidate in an election race). If a regular user has not yet selected a candidate for a particular upcoming election race, the daily action may thus prompt them to do so.
Our users are not compensated or incentivized in any way. They use the app for the user experience in participating in our democracy.
6. Method(s) and Mode(s) of Data Collection
All data is exclusively collected from users who proactively take polling actions (rank a candidate, answer a survey question, vote on a bill, or rate a representative) in the ActiVote app. The app is only available in English.
7. Dates of Data Collection
All user choices within the ActiVote app are registered with a time stamp in our database. For every survey, we select all user choices in a selected time interval (the field period). We report the start, end and median date of the field period of each survey.
8. Sample Sizes and Discussion of Precision of the Results
In every survey report we include the number of participants for that survey. We also provide a measure we call the “average expected error”. The average expected error is based on an analysis of the actual error in ActiVote’s historical election polls per sample size. For the 2024 election cycle, the average expected error is approximately equal to 1.0 times the traditional “MoE” calculation, while our average expected error per primary election poll is approximately equal to 1.5 times that same traditional “MoE” calculation.
The following table shows what this means for various sample sizes.
Thus, for a poll with a sample size of 800, we would expect an average error of about 3.5% in general elections, and 5.2% in primary elections.
These average expected errors are the result of an analysis of over 500 historic general elections polls executed in 2020-2024 and over 300 primary election polls in that same period: our current polling engine, applied to the data of each of those races shows an average error in line with the above table. We believe that this historical performance is the best predictor for future performance. For more details see our accuracy analysis.
We do not have sufficient historical data on our performance with very large sample sizes, however, for the time being we will assume that the same rules will hold until historical data will show otherwise.
No human coders are involved in our surveys.
9. How the Data Were Weighted
For most participants in our surveys, we have a link with their voter registration, which provides us information about the following characteristics: (1) age, (2) gender, (3) party affiliation, (4) ethnicity, (5) income, (6) education, and (7) voting chance based on data in the national voter file from L2.
We also weigh on region, approximately equally split in rural, suburban, and urban. We define this based on the population density per zip code: the 1/3 of the population living in zip codes with the lowest population density are deemed rural, those in the 1/3 living in zip codes with the highest population density are deemed urban. The final 1/3 are deemed suburban.
For each individual state, as well as for the nation, we use those same voter files to determine for each characteristic what percentage of the likely voters it represents. For the general presidential election on November 5th, 2024, the target percentages for likely voters are shown in the two tables below.
Finally, we also weigh on the survey participant’s Political Matrix position. This is an ActiVote-specific measure calculated for each survey participant based on the collection of all survey questions they have answered.
10. How the Data Were Processed and Procedures to Ensure Data Quality
All surveys are automatically processed by ActiVote’s proprietary polling engine, providing polling results in Microsoft Excel tables and graphs ready to publish. This process is nearly fully automated which limits the possibility for human error. Before publishing any poll, the tables and graphs are inspected to determine whether the results pass various “smell-tests” to ensure that the polling engine ran without error.
For all ActiVote users a number of automated checks are performed to determine whether they are believed to be who they say they are. If users are detected that are believed to not be genuine, they are excluded from the poll. We cannot provide details on which checks alert us to non-genuine users, given the opt-in nature of our surveys, as that would provide such users with information on how to bypass our detection mechanisms.
11. A General Statement Acknowledging Limitations of the Design and Data Collection
ActiVote’s polls are based on an opt-in app panel, meaning that anyone can participate by choice, which provides us with a non-probability sample. As a result, in each of our polls a particular subgroup of the overall electorate could be overrepresented.
Our polls aim to correct for such overrepresentation by weighing on various criteria, such as age, gender, party affiliation, ethnicity, income, education, voting chance, region and Political Matrix position. Despite these steps to come to a representative poll, some subgroups may be underrepresented, potentially skewing the results in an unknown way.
