Analyzing Margins of Error and Coefficients of Variation

Data from the American Community Survey (ACS), as well as any other data source that is based on a statistical sample rather than a full census of the entire population, has an associated measure of reliability. Mapping this type of data requires assessing the appropriate geographic resolution (or unit of display) in order to ensure an acceptable level of accuracy based on the sampling strategy. Reliability measures include:

For more information see:

Michael Burns, “Understanding the Margin of Errors and the Coefficient of Variance in the American Community Survey”, U.S. Census Bureau. (PDF)
Ben Horowitz, “ACS Mapping Methodology: How to Spatially Display ACS 5-Year Data”, Greater New Orleans Community Data Center. (PDF)
Ben Horwitz, “Geographies of Poverty: Improving the Reliability and Usability of Spatial Displays of Small Area Data from the American Community Survey”, The Data Center. (PDF)
Min Sun and David W.S. Wong. 2010. "Incorporating Data Quality Information in Mapping American Community Survey Data." Cartography and Geographic Information Science, Vol. 37, No. 4, p. 285. (PDF)

Margin of Error (MOE) – a measure of the precision of the data. The MOE reflects the estimated range within which the true value is likely to fall. The MOE is calculated using a series of statistical equations based on the size of the sample and the expected normal distribution of values. The data provider usually calculates the MOE and includes the figures as part of the data tables. The MOE is indicated as a +/- value. For example, a value of 80 with a MOE of +/- 20 means that the true value could fall anywhere between 60 and 100. An acceptable level of reliability of spatially referenced data is difficult to ascertain through the MOE alone, since it can vary considerably between geographic units such as census tracts.
Coefficient of Variation (CV) – the relative amount of sampling error associated with an estimate. The CV is a standardized measure of reliability expressed as a percentage. Data users can compute a CV directly from the MOE. To calculate a CV (at a 90% confidence level), use the following equation: CV= ((MOE/1.645)/Estimate)*100. For example, if you have an estimate of 80 +/- 20, the CV for the estimate is 15.2% (the sampling error represents slightly more than 15% of the estimate). What constitutes an acceptable CV depends on the intended use of the data and what level of accuracy is deemed necessary to support decisions based on the data.

Example: The ACS normally compiles data based on a county-level statistical sample. Care must be taken in displaying this data at finer spatial resolutions, such as census tracts. Disaggregating ACS data into increasingly finer levels of analysis is also problematic

To illustrate, an analysis of the MOEs and CVs for the ACS data (PDF) included in the Regional Equity Atlas 2.0 indicated that only some of the ACS data could be mapped at the census tract level because the majority of the CVs for the tracts were considered too high – or marginal at best – at this level of geographic resolution or data disaggregation. Much of the ACS data could only be mapped at the PUMA level (Public Use Microdata Area), a census geography unit that is much larger than a census tract but somewhat smaller than a county.

The map to the right shows the CV values for census tract- level data based on 2011 ACS 5-year estimates of the veteran population. Green indicates acceptable, orange indicates marginal and red indicates unacceptable. Based on this analysis, CLF concluded that it could only map these data at the PUMA level.