The steep rise in U.S. criminal punishment in recent decades has spurred scholarship on the collateral consequences of imprisonment for individuals, families, and communities. Several excellent studies have estimated the number of people who have been incarcerated and the collateral consequences they face, but far less is known about the size and scope of the total U.S. population with felony convictions beyond prison walls, including those who serve their sentences on probation or in jail. This article develops state-level estimates based on demographic life tables and extends previous national estimates of the number of people with felony convictions to 2010. We estimate that 3 % of the total U.S. adult population and 15 % of the African American adult male population has ever been to prison; people with felony convictions account for 8 % of all adults and 33 % of the African American adult male population. We discuss the far-reaching consequences of the spatial concentration and immense growth of these groups since 1980.