Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

ACKNOWLEDGMENT This work was sponsored by the American Association of State Highway and Transportation Officials (AASHTO), in cooperation with the Federal Highway Administration, and was conducted in the National Cooperative Highway Research Program (NCHRP), which is administered by the Transportation Research Board (TRB) of the National Academies. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, FAA, FHWA, FMCSA, FTA, Transit Development Corporation, or AOC endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. DISCLAIMER The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research. They are not necessarily those of the Transportation Research Board, the National Research Council, or the program sponsors. The information contained in this document was taken directly from the submission of the author(s). This material has not been edited by TRB.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. On the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, on its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. The Transportation Research Board is one of six major divisions of the National Research Council. The mission of the Transporta- tion Research Board is to provide leadership in transportation innovation and progress through research and information exchange, conducted within a setting that is objective, interdisciplinary, and multimodal. The Board’s varied activities annually engage about 7,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individu- als interested in the development of transportation. www.TRB.org www.national-academies.org

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules i Table of Contents CONTENTS PAGE Table of Contents ...................................................................................................................................... i List of Figures and Tables ...................................................................................................................... iv Author Acknowledgments ..................................................................................................................... vii Abstract .................................................................................................................................................. viii Executive Summary ........................................................................................................................... ES-1 1. Initial Investigatory Research .................................................................................... ES-1 2. Development Phase .................................................................................................... ES-4 3. Validation Phase ........................................................................................................ ES-8 1. Introduction to Research Investigations ................................................................................ 1-1 1.1 Census DRB Rules on ACS Five-Year Tabulations and Disclosure Risk Elements in CTPP Tables ..................................................................................... 1-2 1.1.1 General Structure of CTPP Tables ................................................................... 1-2 1.1.2 Disclosure Rules .............................................................................................. 1-4 1.1.3 Risk Elements .................................................................................................. 1-6 1.1.4 Addressing the Risk Elements ......................................................................... 1-7 1.2 Transportation Planning Considerations ..................................................................... 1-13 1.2.1 Test Sites ........................................................................................................ 1-15 1.2.2 Usability Ratings............................................................................................ 1-16 1.2.3 Data Consistency Issues ................................................................................. 1-16 1.3 ACS Operations Considerations ................................................................................. 1-16 1.3.1 Input Datasets ................................................................................................ 1-16 1.3.2 Operations Timeline/Resources ..................................................................... 1-17 1.3.3 Workplace Allocation .................................................................................... 1-17 1.4 Initial Critical Assessment of Promising Disclosure Avoidance Techniques ............. 1-18 1.4.1 Set of Criteria ................................................................................................. 1-18 1.4.2 Assessment of Initial Approaches .................................................................. 1-19 1.4.2.1 Deterministic Approaches ............................................................. 1-19 1.4.2.2 Perturbation Approaches ............................................................... 1-20 1.4.3 Credible Approaches Selected for the Development Phase ........................... 1-23 2. Development Phase .................................................................................................................. 2-1 2.1 Details of the Perturbation Approaches ........................................................................ 2-3 2.1.1 Evaluation Design ............................................................................................ 2-3 2.1.2 Preliminary Steps ............................................................................................. 2-3 2.1.3 Perturbation Approaches .................................................................................. 2-6 2.1.3.1 Semi-Parametric .............................................................................. 2-8 2.1.3.2 Parametric ...................................................................................... 2-14 2.1.3.3 Constrained Hot Deck ................................................................... 2-18 2.1.4 Weight Calibration ......................................................................................... 2-22 2.1.5 Variance Estimation ....................................................................................... 2-24

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules ii 2.2 Data Utility and Disclosure Risk Measures ................................................................ 2-26 2.2.1 Data Utility Measures .................................................................................... 2-27 2.2.1.1 CTPP and ACS Data Elements That Affect the Formulation of Data Utility Measures ........................................... 2-27 2.2.1.2 Quantifying the Impact on Data Utility ......................................... 2-28 2.2.1.3 Comparison with Home-Based Work Outputs .............................. 2-36 2.2.2 Disclosure Risk Measures .............................................................................. 2-45 2.3 Recommendation for the Validation Phase ................................................................. 2-48 3. Validation Phase ...................................................................................................................... 3-1 3.1 Perturbation Approach Applied During the Validation Phase ...................................... 3-3 3.1.1 Initial Risk Analysis......................................................................................... 3-5 3.1.2 Data Replacement ............................................................................................ 3-7 3.1.2.1 Semi-Parametric ............................................................................ 3-10 3.1.2.2 Constrained Hot deck .................................................................... 3-12 3.1.3 Weight Calibration ......................................................................................... 3-14 3.1.4 Variance Estimation ....................................................................................... 3-15 3.2 Impact on Data Utility and Disclosure Risk ............................................................... 3-17 3.2.1 Impact on Data Utility ................................................................................... 3-17 3.2.2 Disclosure Risk Measures .............................................................................. 3-34 4. Production Run Processing ..................................................................................................... 4-1 4.1 Introduction to Processing Steps ................................................................................... 4-1 4.2 Program Components ................................................................................................... 4-3 4.2.1 Program Component: Initial Risk Analysis ..................................................... 4-3 4.2.2 Program Component: Data Replacement......................................................... 4-7 4.2.3 Program Component: Raking ........................................................................ 4-10 4.2.4 Program Component: Utility Measures ......................................................... 4-11 4.2.5 Program Component: Risk Measures ............................................................ 4-13 4.2.6 Program Component: Cleanup ....................................................................... 4-14 4.3 Other topics relating to the processing runs ................................................................ 4-15 4.3.1 Set A and Set B tables .................................................................................... 4-15 4.3.2 Variance Estimation for Set B Tables ............................................................ 4-16 4.3.3 Impact of Adding Tables ............................................................................... 4-16 References ............................................................................................................................................. R-1

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules iii Appendices APPENDIX PAGE A Census Transportation Planning Products (CTPP) ................................................................... A-1 B Traffic Analysis Zone Estimates from2006-2008 ACS Data, By State .................................... B-1 C Set B Tables .............................................................................................................................. C-1 D Set of Predictor Variables ......................................................................................................... D-1 E Development Phase: Tabular Summary Results for the Data Utility Measures ....................... E-1 F Development Phase: Plots of ACS and Perturbed Data for Mean Travel Time ....................... F-1 G Development Phase: Plots of ACS and Perturbed Data for Mean Household (HH) Income ............................................................................................................................. G-1 H Development Phase: Plots of ACS and Perturbed Data for Weighted Counts – Full Replacement ................................................................................................................... H-1 I Development Phase: Plots of ACS and Constrained Hot Deck Data for Weighted Counts – Partial Replacement .................................................................................. I-1 J Development Phase: Plots of ACS and Perturbed Data for Pairwise Correlations ................................................................................................................................ J-1 K Development Phase: Tables For Travel Model Output Comparisons ...................................... K-1 L Development Phase: Figures For Travel Model Output Comparisons ..................................... L-1 M Validation Phase: Tabular Summary Results for the Data Utility Measures ........................... M-1 N Validation Phase: Plots of ACS and Perturbed Data for Mean Travel Time ............................ N-1 O Validation Phase: Plots of ACS and Perturbed Data for Mean Household (HH) Income ............................................................................................................................. O-1 P Validation Phase: Plots of ACS Partially Perturbed Data for Weighted Counts ...................................................................................................................................... P-1 Q Validation Phase: Plots of ACS and Perturbed Data for Pairwise Correlations ....................... Q-1 R Validation Phase: Tables For Travel Model Output Comparisons ........................................... R-1 S Validation Phase: Figure S-s For Travel Model Output Comparisons ..................................... S-1 T SAS Code for Driver Programs ................................................................................................ T-1 U List of SAS Programs ............................................................................................................... U-1

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules iv List of Figures and Tables FIGURE PAGE ES-1. Bubble Plots of ACS and Partial Perturbed Mean Travel Time for Iowa’s TAZs: Left: Constrained ot deck; Middle: Semi-Parametric; Right: Parametric (n ≥30) .................................................................................................................. ES-6 ES-2. Bubble Plots of ACS and Partial Perturbed Mean Travel Time for Atlanta’s TAZs: Left: Development Phase, Right: Validation Phase (n≥30) ....................................... ES-10 1-1. Scatter Plot by State of TAZ to Block Group Ratio and Percentage of ACS Sample in TAZs Below DRB Thresholds ................................................................................ 1-10 1-2. Scatter Plot by State of TAZ to Block Group Ratio and Percentage of ACS Sample in CTAZ50 Below DRB Thresholds ........................................................................... 1-10 1-3. Scatter Plot by State of TAZ to Block Group Ratio and Percentage of ACS Sample in CTAZ300 Below DRB Thresholds ......................................................................... 1-11 2-1. Development Phase: CTPP Research Approach ........................................................................ 2-2 2-2. Development Phase: Semi-Parametric Approach Flowchart ..................................................... 2-9 2-3. Development Phase: Parametric Modeling Approach Flowchart ............................................ 2-15 2-4. Development Phase: Illustration of Bin Formation ................................................................. 2-20 2-5. Development Phase: Estimated Standard Errors of the County-Level Mean Travel Time for Workers Who Drove Alone: ACS 2006-2008 ............................................... 2-26 3-1. Validation Phase: CTPP Research Approach ............................................................................ 3-4 3-2. Validation Phase: Data Replacement Step Processing Flow ..................................................... 3-7 3-3. Validation Phase: Estimated Standard Errors of the County-Level Mean Travel Time (in minutes) for Workers Who Drove Alone: ACS 2005-2009 .......................... 3-16 4-1. Overall Perturbation Process Flowchart .................................................................................... 4-2 4-2. Flowchart of Creation of CTAZs and CTAZ-Level Covariates Program Component ................................................................................................................................. 4-4 4-3. Flowchart of Person-Level Initial Risk Analysis Program Component ..................................... 4-5 4-4. Flowchart of Household Level Initial Risk Analysis Program Component .............................. 4-6 4-5. Flowchart of Data Replacement Program Component .............................................................. 4-8 4-6. An example of a Master Index File ............................................................................................ 4-9 4-7. Flowchart of Household Level and Person Level Control Total Calculations and Raking Program Component ............................................................................................. 4-11 4-8. Flowchart of Data Utility Measures Program Component ...................................................... 4-12 4-9. Flowchart of Disclosure Risk Measures. ................................................................................. 4-14

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules v TABLE PAGE ES-1. Statistical Disclosure Control Treatment Options Evaluated for CTPP ................................. ES-3 ES-2. Validation phase: Raking dimensions for the Household File ................................................ ES-8 ES-3. Validation phase: Raking dimensions for the Person File ...................................................... ES-9 1-1. CTPP Tables with Cell Aggregates, Means, and Medians ........................................................ 1-4 1-2. Disclosure Rules for CTPP Tables Based on the Five-Year ACS ............................................. 1-5 1-3. Comparison of ACS and Census 2000 Long Form Sample Sizes ............................................. 1-7 1-4. Comparison of Percent of Records from Housing Units in TAZs and Collapsed TAZs that Contain a DRB Rules Violation in at Least One Table: ACS 2005-2009 ......................................................................................................................... 1-9 1-5. Comparison of Percent of Records from Group Quarters in TAZs and Collapsed TAZs that Contain a DRB Rules Violation in at Least One Table: ACS 2005-2009 ......................................................................................................................... 1-9 1-6. CTPP Variables and Their Usability and Identifiability Levels .............................................. 1-12 1-7. Model and ACS Comparison Test Sites .................................................................................. 1-15 1-8. List of Statistical Disclosure Control Treatments Initially Planned......................................... 1-19 2-1. Development Phase: Subset of Preliminary Variables Perturbed .............................................. 2-8 2-2. Development Phase: FORCELIST Variables for Each Dependent Variable .......................... 2-11 2-3. Development Phase: Number of Prediction Groups and Weights Cell for Each Variable Perturbed .......................................................................................................... 2-12 2-4. Development Phase: Published Categories of Travel Time (illustrative) ................................ 2-19 2-5. Development Phase: Raking Dimensions for the Household File ........................................... 2-23 2-6. Development Phase: Raking Dimensions for the Person File ................................................. 2-23 2-7. Development Phase: Percentiles of the Raking Factors: Person Level .................................... 2-24 2-8. Development Phase: Percentiles of the Raking Factors: Household Level ............................. 2-24 2-9. Development Phase: Proportion of TAZs with 30 or More ACS Cases, Three-Year ACS ...................................................................................................................... 2-30 2-10. Development Phase: Distribution of Absolute Relative Differences for Mean Travel Time at the TAZ Level by Mean Travel Time, 3-Year ACS ....................................... 2-30 2-11. Development Phase: Distribution of Absolute Relative Differences for Mean HH Income at the TAZ Level by Mean HH Income, Three-Year ACS .................................. 2-31 2-12. Development Phase: Tests for Comparison of Travel Demand Model Output and Raw and Perturbed ACS Data (and Direct Comparison of Raw and Perturbed ACS) and Location of Data Tables in Appendix K ................................................. 2-37 3-1. Validation Phase: Subset of Preliminary Variables that were Perturbed ................................... 3-9 3-2. Validation Phase: FORCELIST Variables for Each Dependent Variable ............................... 3-11 3-3. Validation Phase: Number of Prediction Groups and Weights Cell for Each Perturbed Variable ................................................................................................................... 3-11 3-4. Validation Phase: Raking Dimensions for the Household File ................................................ 3-14 3-5. Validation Phase: Raking Dimensions for the Person File ...................................................... 3-14

NCHRP Project 08-79 Final Report: Producing Transportation Data Products from the American Community Survey That Comply With Disclosure Rules vi 3-6. Validation Phase: Percentiles of the Raking Factors: Person Level ........................................ 3-15 3-7. Validation Phase: Percentiles of the Raking Factors: Household Level .................................. 3-15 3-8. Validation Phase: Percentage of TAZs by ACS Sample Size Categories ............................... 3-19 3-9. Validation Phase: Distribution of Absolute Relative Differences for Mean Travel Time at the TAZ Level by Mean Travel Time, Five-Year ACS .................................. 3-19 3-10. Validation Phase: Preliminary Simulation Results on Tests for Potential Bias in Drive Alone Mean Travel Time in Olympia, by CTAZ300 ................................................ 3-20 3-11. Validation Phase: Distribution of Absolute Relative Differences for Mean Household Income at the TAZ Level by Mean Household Income, Five-Year ACS .......................................................................................................................................... 3-20 3-12. Validation Phase: Relative Difference Between ACS and Perturbed Estimates, and Ratios of Standard Errors from (f5) and (f3) to that from Usual ACS Estimator, by CTAZ. ............................................................................................ 3-23 3-13. Validation Phase: Coverage Rates of Confidence Intervals Based on Three Variance Estimators and 95% Confidence Interval Formulae. ................................................ 3-24 3-14. Validation Phase: Median of Ratios of Standard Errors for Weighted Cell Counts ...................................................................................................................................... 3-25 3-15. Validation Phase: Tests for Comparison of Travel Demand Model Output and Raw and Perturbed ACS Data (and Direct Comparison of Raw and Perturbed ACS) and Location of Data Tables in Appendix R ................................................. 3-31 4-1. Initial Risk Analysis: Difference Between Input and Output Datasets ...................................... 4-3 4-2. Data Replacement: Difference Between Input and Output Datasets ....................................... 4-10 4-3. Raking: Difference Between Input and Output Datasets ......................................................... 4-10 4-4. Cleanup: Difference Between Input and Output Datasets ....................................................... 4-15