Is the 2020 Census “Fit-for-apportionment”?
Written by: Roger Hoerl and Thomas C. Redman
In a press release dated March 10, 2022, Census Director Robert Santos stated “Taking today’s findings as a whole, we believe the 2020 census data are fit for many uses in decision-making as well as for painting a vivid portrait of our nation’s people.” The “fit for many uses” clause leads to several questions; “for what uses is the census fit?,” “for what purposes is it not?,” and perhaps most importantly, “was it fit for apportionment?” Apportionment is the critically important task of allocating congressional representatives to the fifty states. We performed an independent quality assurance review using data available on the Census Bureau website to address the last question. Our review raises serious concerns about the fitness of the 2020 census for apportionment. In particular, six states appear to have been assigned the wrong numbers of representatives.
Background: Quality assurance (QA) provides an independent assessment of the overall impact of a quality program with emphasis on the question, “did it provide the required quality?” It can be a complicated question, particularly for the census, which is used for many purposes. For apportionment, with its high political consequences, we expect the census to:
- Ensure that each state is assigned the right number of representatives, AND
- Provide a measure of comfort that results can be trusted.
These criteria constitute a high bar. But the stakes are high, fully justifying the expectations.
Our review is straightforward– we collected relevant data from census.gov, including “apportionment data,” the population count used for apportionment, and estimated miscount rates, based on a Census-conducted Post Enumeration Survey (PES). While the apportionment state populations are counts, performed according to the US Constitution and Census regulations, the PES is a statistically-based survey used to evaluate the accuracy of the apportionment count. One hopes that the resulting estimates of miscount rates are small.
Next, we counted “large” estimated miscounts, defined in three ways, and calculated summary statistics. We compared the numbers of representatives assigned to each state using apportionment data (the official results) with projected results incorporating estimated miscounts from the PES survey. We based these analyses on the algorithm included on the Census website, found here. We then collected data and performed similar analyses using the 2010 census as a basis for comparison. Finally, we organized our results into Tables 1 and 2, presented in the Appendix, and Figure 1, below.
To meet our first criterion “fitness-for-apportionment,” numbers of representatives assigned to each state using apportionment and the estimated (from PES) state populations should agree. Low miscount rates, and low state-to-state variation could provide the comfort called for in our second criterion.
Results: Any of the statistics presented in Table 1 should lead one to question the fitness-for-apportionment of the 2020 census. There were statistically significant miscounts in fourteen states and even one should give pause (Note: The Census Bureau reported these results on May 19, 2022). Fourteen is of enormous concern, especially when compared to zero statistically significant miscounts in 2010. Other results in Table 1 amplify this concern. Importantly, Table 2 shows that six states (Colorado, Minnesota, Rhode Island, Florida, Texas, and Tennessee) receive different numbers of representatives when the estimated corrections are taken into account, i.e., when using the PES population instead of the apportionment population.
Had each state simply been over- or undercounted by a similar proportion, apportionment issues would likely have been minor. But the state-to-state variation is large, especially compared to the 2010 census. There were twenty-one states with large miscounts (see Table 1) with ten states having large undercounts and eleven large overcounts.
This point may be easier to appreciate by looking at Figure 1.
In 2010, estimated miscount rates are tightly clustered near zero, while for 2020 they are much more dispersed. Ultimately, the high state-to-state variation and high miscount rates led to the different numbers of representatives officially apportioned and those projected using PES estimated state populations.
Perhaps more than anyone, we appreciate the leadership, management, and operational challenges associated with data quality. Conducting a census is difficult (try simply counting your socks!) and the pandemic added enormous challenges. One should expect problems! Interestingly, the UK also experienced serious problems conducting its 2021 census, but frankly we’re surprised this topic hasn’t generated far more discussion.
Our two “fit-for-apportionment” criteria represent a high bar, but in our view, the pandemic does not justify relaxing them. One cannot easily escape our main conclusion–that the 2020 Census is not “fit-for-apportionment.” It appears to fail criterion 1: assigning the correct number of representatives to each state, and it most certainly fails criterion 2: providing confidence in the results.
Appendix: Tables and Supporting Definitions
Definitions/notes relevant to the tables above: Apportionment population (X): The population total, for State X (or the entire country) used for apportionment. (Estimated) Corrected population (X): The estimated population total, for State X (or the entire country), based on the Post Enumeration Survey (PES).
(Estimated) Miscount (X): The apportionment population minus the estimated (PES) total, for State X (or the entire country). If the miscount is less than zero, the miscount is called an undercount; if greater than zero, an overcount.
(Estimated) Miscount rate (X): The (Estimated) Miscount divided by the (Estimated) corrected (PES) population, for State X (or for the entire country).
Standard error/Root mean Square error (X): The estimated error (uncertainty) in the Post Enumeration Survey, for State X (Standard error for 2020, Root mean square error for 2010).
Statistically significant differences: The count of States whose (estimated) miscount rates are judged statistically significant.
NOTE 1: We sourced apportionment populations, estimated miscount rates, statistically significant differences and standard errors/root mean square errors for the 2020 and 2010 censuses from sources on the census.gov website. We used these to calculate the (estimated) corrected populations and (estimated) miscounts and the statistics below.
Large (estimated) miscounts: The count of states whose (estimated) miscount rates are judged “large” by our (current authors’) criteria:
· The (estimated) miscount rate is greater than two percent in absolute value.
· The (estimated) miscount is greater than 100,000 people in absolute value.
Egregiously large (estimated) miscounts: The count of States whose estimated miscounts is greater than 500,000 in absolute value. This criterion is also from the current authors’.
NOTE 2: We recognize that our cut-off points are somewhat arbitrary.
NOTE 3: In later work, we may wish to define a “large (estimated) miscount (or miscount rate), as the minimum that would change the number of representatives assigned to a State. That is beyond this QA exercise, however.
Out-of-control: We judge the counting process for a state as “out-of-control” when the (estimated) miscount rate/divided by its standard error/root mean square error exceeds three (3) in absolute value.
Population rank (X): In a largest to smallest ordering of populations, the rank assigned State X.
Changes in population rank: The count of states whose population rank is different when using apportionment and (estimated) corrected populations from the PES.
Number of representatives (X): The total number of representatives assigned to a state.
Changes in number of representatives: The count of states whose number of representatives is different when using apportionment and (estimated) corrected populations from the PES.
Net apportionment miscount rate: The sum of the absolute values of (estimated) state miscounts divided by the (estimated) total US population.
Standard Deviation of miscount rates: The standard deviation of (estimated) miscount rates of the 50 states.
About the Authors:
Roger Hoerl teaches statistics at Union College in Schenectady, NY. Previously, he led the Applied Statistics Lab at GE Global Research.
Thomas C. Redman“the Data Doc,” helps organizations chart courses to data-driven futures, with special emphasis on quality and data science.