6

Measuring Criminal Justice Contact and Incarceration Experience in Health Surveys

Michael Schober (New School for Social Research) began the workshop session by pointing out that each of the five health surveys that will be discussed has its own core functions and constituencies whose information needs it specifically addresses. There is wide variety in the sampling processes, sample sizes, administration modes, and areas of measurement across the surveys. He acknowledged that inclusion of criminal justice involvement measures on these surveys can have implications for instrument length, respondent burden, and, ultimately, response rates, which are already a concern for some surveys. Schober asked the workshop participants to also bear in mind that because of the large variation among the surveys, a singular set of proposed changes may not be feasible.

THE NATIONAL HEALTH INTERVIEW SURVEY

Marcie Cynamon (National Center for Health Statistics [NCHS]) provided an in-depth overview of the National Health Interview Survey (NHIS).

NHIS is one of the largest and oldest health surveys overseen by the U.S. Department of Health and Human Services.¹ The survey is aimed at collecting and analyzing data on a broad range of health topics to monitor the health of the non-institutionalized civilian population. The NHIS is an address-based multistage probability sample that collects data from 75,000 to 100,000 individuals in 35,000 to 40,000 households. There are

___________________

¹ The U.S. Census Bureau serves as the data collection agent for this survey.

at least two primary sampling units in each state, which enables production of state-level estimates on a wide range of variables. NHIS is primarily administered in person in respondents’ homes, with roughly 22 percent of the interviews completed by telephone.

Since the survey began in 1957, Cynamon noted, NCHS has produced public use microdata files and analytic reports. More detailed information that cannot be released on public use files due to disclosure risk may be available via the NCHS Research Data Center. The NHIS data are linked to other data sources, such as the National Death Index, and to data files from other agencies, such as the Centers for Medicare and Medicaid Services and the U.S. Department of Housing and Urban Development.

Cynamon explained that since 1997 the survey has included three modules, each directed toward different component in a household: the adult(s), the child(ren), and the entire family. The first 20 to 30 minutes are a family-level interview in which information about the entire household is collected and a knowledgeable respondent answers some health questions about everyone in the household. The next two segments are directed to one adult and one child (if there are children in the household), who are randomly selected, for a more detailed interview, each approximately 20 minutes long. In addition, the questionnaires have supplements that can pertain to the sample adult or, in some cases, to the sample child, which add another 20 to 30 minutes to a respondent’s total interview time.

Currently, Cynamon explained, the survey is being redesigned to improve measurement of covered health topics, reduce respondent burden, harmonize overlapping content with other federal health surveys, establish a long-term structure of ongoing and periodic topics, and incorporate advances in survey methodology and measurement. As part of that redesign, NCHS plans to incorporate biomeasures through a passive monitoring approach. The redesigned questionnaire will be finalized by early 2017 and fielded in 2018. Outreach activities to survey users and professional associations have generated many suggestions on rearrangement of the survey, which the agency has accommodated to the best of its capacity.

Cynamon described how the trends in questionnaire length and response rates in the survey’s three cores are moving in opposite directions. Declining response rates affect the reliability and validity of NHIS estimates. Instead of having an entire family module, the proposed new structure of the survey will have a brief set of questions about the family, framed in a manner to collect basic demographics about a household that will aid in selection of a sample adult and sample child. The focus of the interview will now be on the sample adult and sample child, reflecting that 95 percent of NHIS publications show data from these individual levels. The new adult core module contains information about family income and program participation. The content of the child core is still under development. However, the

redesigned survey will still contain annually rotating core modules about additional household content, such as tobacco use and specific health conditions, and certain federal agencies will also have the opportunity to sponsor 5-minute-long supplemental modules.

Cynamon described the decision criteria of inclusion of questions or topics in the core module:

• leading causes of morbidity and mortality;
• health insurance coverage;
• health care access and utilization;
• targets of major federal health promotion initiatives;
• behavioral risk or protective factors for the above relevant content areas;
• other factors that identify populations most at risk for or protected from the above relevant content areas (e.g., family income); and
• intermediate health outcomes for the above relevant content areas.

THE BEHAVIORAL RISK FACTORS SURVEILLANCE SYSTEM

Wayne Giles (Centers for Disease Control and Prevention [CDC]) gave an overview of the Behavioral Risk Factors Surveillance System (BRFSS), which is a system of state-based health surveys. The BRFSS has been in existence since 1984; it arose from a need for public health data at state and county levels. He noted that since the implementation of public health programs and health promotion often takes place at the state and county levels, it was recognized that there is a need for health surveillance data at those levels. The BRFSS is an outcome of a joint partnership between CDC and states, with the content decided jointly as well.

The state administers telephone surveys by both landlines and cell phones. More than 450,000 interviews are completed each year. On average, 35 percent of the interviews take place on cell phones, but in some states the rate is more than 60 percent. Survey respondents are asked about health conditions, health behaviors, and health care utilization. The BRFSS response rates for combined landline and cell phone interviews range between 42 and 80 percent (the median is at 67 percent). Giles noted that it is a challenge to get people to answer the phone, but once they answer, BRFSS interviewers are generally successful in conducting an interview. He also noted that interviewers have held people’s interest on cell phones longer than on landlines.

Giles described the four components of the BRFSS. The core survey includes annual and rotating core sections and is uniformly fielded in all states and territories. Questions are asked every year on the core survey topics: HIV/AIDS, diabetes, asthma, cardiovascular disease, alcohol con-

sumption, health status, health care access, healthy days, emotional support and life satisfaction, disability, tobacco use, and sleep. There are also rotating core topics. In even-numbered years the topics include breast and cervical cancer screening, prostate screening, colorectal cancer screening, oral health, falls, seatbelt use, and drinking and driving. In odd-numbered years, the topics are fruit and vegetable consumption, hypertension awareness, cholesterol awareness, arthritis burden, and physical activity. Socioeconomic characteristics and demographic information are also collected.

The BFRSS also fields supplemental modules that are generally proposed by CDC programs and other agencies, including the Substance Abuse and Mental Health Services Administration and the U.S. Department of Veterans Affairs. States also include specific questions to meet their individual needs and issues. Giles said that the BRFSS also allows for inclusion of special project questions, such as H1N1 flu. These add-on special project questions are included on an as-needed basis, generally in response to requests from various programs and agencies. Giles said that data from the BRFSS are used extensively by the federal and state health departments, policy makers, universities, researchers and research organizations, media, insurance companies, and the general public. One primary use is in the publication America’s Health Rankings, which ranks states and counties on their core measures of health and health outcomes. Giles also mentioned a 500-city partnership project between the CDC Foundation and the Robert Wood Johnson Foundation to produce, analyze, and release census-tract-level health data compiled from multiple sources that will produce health outcomes data for neighborhoods and cities.

THE NATIONAL SURVEY OF DRUG USE AND HEALTH

Arthur Hughes (Substance Abuse and Mental Health Services Administration [SAMHSA]) described the design, sampling frame, instruments, and response rates of the National Survey of Drug Use and Health (NSDUH) and discussed the potential of adding a criminal justice module to the survey. The NSDUH estimates are the primary source of statistical information on substance use and mental health issues in the U.S. civilian non-institutionalized population. The survey is sponsored by SAMHSA and planned and managed by the SAMHSA Center for Behavioral Health Statistics and Quality, with data collected by RTI International. The NSDUH estimates are produced at national, state, and below-state levels, which helps determine substance use and mental health service needs and allocation of funding to meet those needs. The data are useful in gauging trends of substance use, substance use disorders, and mental health problems in the general population. The survey also collects information on such covariates as arrests, probation, and parole.

Hughes explained that the NSDUH is designed to be a multistage area probability sample of households. States are the first level of stratification, followed by stratification within each state, then census tracts, census block groups, and segments consisting of a collection of census blocks are selected within each state stratum. The 12 largest states have an annual target sample of over 1,500 each, while the 38 other smaller states plus the District of Columbia have a sample size of 960 each, which brings the total sample size to approximately 67,500 people. There are several different stages in the sample selection process. Geographic selection of segments is conducted in stage three, and these areas are canvassed to generate a list of dwellings (addresses/households). Survey statisticians select the sample dwelling units from the canvassed list. Once a dwelling is selected, a professional RTI interviewer conducts a screening interview, and then zero, one, or two people aged 12 and above are selected from the household for an in-person interview. The survey oversamples people aged 12-25. The response rate for the 2014 NSDUH screener was 82 percent, and the response rate for the in-person interview was 71 percent. The NSDUH relies on a computer-assisted personal interview (CAPI) for collecting demographic variables and on an audio computer-assisted self-interview (ACASI) for collecting data on recency, frequency, and initiation of substance use.

Hughes told the workshop participants that questions about criminal justice involvement were added in 1991 to address the paucity of data on the relationship between drug use and criminal behavior among the general population. These questions are part of the special topics module, which is administered to respondents aged 12 and older. This NSDUH module asks whether the respondent has ever been arrested and booked for breaking a law. If the answer is yes, the respondent is then asked how many times he or she has been arrested, and the respondent is also asked questions about the type of offense. The reference period for all of the criminal justice questions is the past 12 months.

The NSDUH module covers 17 offenses:

• motor vehicle theft;
• larceny or theft;
• burglary or breaking and entering;
• aggravated assault;
• other assault, such as simple assault or battery;
• robbery;
• forcible rape;
• murder, homicide, or non-negligent manslaughter;
• arson;
• driving under the influence of alcohol or drugs;
• drunkenness or other liquor law violations;

• possession of tobacco (only for respondents aged 12-17);
• possession, manufacture, or sale of drugs;
• prostitution or commercialized sex;
• any other sexual offense, not including rape or prostitution;
• fraud, possessing stolen goods, or vandalism; and
• some other offense.

The survey also includes questions on probation, parole, supervised release, and other conditional release with a recency period of 12 months.

Hughes reported that from 2008 to 2014 the average weighted prevalence estimate of respondents who had ever been arrested was 16.8 percent. For probation the prevalence estimate was 2.2 percent, and for parole/ supervised release it was 0.7 percent. The total sample size of ever arrested over the same 7-year period was 67,000. The numbers for on probation and on parole/supervised release were 15,000 and 4,400, respectively. Less than 1 percent of data are missing for these variables. The average questionnaire time of NSDUH 2015 was 58.7 minutes for all respondents. However, the response time was 63.9 minutes for respondents who answered yes to the probation question, and was longest for respondents who answered yes to the parole/supervised release question: 66.4 minutes.

Hughes discussed the need to be mindful of question placement when considering adding additional criminal justice questions to a survey like the NSDUH. Currently, the NSDUH does not collect any data related to criminal justice other than prevalence and timing. Adding questions at the end of the special topic module would be difficult given the structure of the interview, which moves from CAPI to ACASI then back to CAPI and identification of proxy respondents for selected questions. Adding questions elsewhere may produce context effect to other questions, including sensitive back-end CAPI questions.

Hughes also noted that placement of new questions on the interview would also undoubtedly produce context effects. And since the population of interest among the NSDUH respondents is small, field tests may not accurately detect any such effects and their consequences. Moreover, since the survey asks sensitive questions, additional questions may raise concerns about approval from an institutional review board (IRB).² He suggested that the optimal approach toward implementing a new module on criminal justice would be to incorporate it into the next NSDUH redesign given the assumption there will be an understanding that the overall change will create a new baseline.

___________________

² An IRB has to approve any sensitive survey questions that involve human subjects (respondents).

THE NATIONAL SURVEY OF FAMILY GROWTH

Anjani Chandra (National Center for Health Statistics) provided a brief description of the National Survey of Family Growth (NSFG) and discussed the incarceration questions it includes for male respondents.

The NSFG came under the federal statistical system due to a portion of the Public Health Service Act, which stipulated that the agency should collect statistics on family formation, growth, and dissolution. The survey is primarily funded by NCHS, the National Institute of Child Health and Human Development, and the Office of Population Affairs, all in the U.S. Department of Health and Human Services. The primary purpose of the survey, which has been conducted since 1973, is to explain variation in birth rates. Thus the survey context is organized around proximate determinants of fertility, looking at factors that directly affect the occurrence of sexual activity, the occurrence of pregnancy from sexual activity, and the occurrence of a live birth from pregnancy. The survey includes items on sexual activity, contraception, infertility, pregnancy loss, and other topics that would affect occurrence of a birth. The NSFG also collects information on covariates such as demographic characteristics and socioeconomic factors.

Chandra explained that the first two rounds of the survey were targeted at ever-married women; then, in 1982, the survey expanded to encompass all women aged 15-44. In 2002, the survey scope was broadened to include men of the same age range, and the content was also expanded to include risk factors for HIV, sexually transmitted diseases, and nonmarital fertility. In addition to complete pregnancy histories and marriage and divorce information, the survey collects data on the roles of fathers in raising children and attitudes about sex, marriage, and parenthood with an aim to understand and describe family formation, growth, and dissolution. In 2015 the age range was expanded to 15-49.

Chandra pointed out that in its current form the NSFG is not a survey of couples: the sample of men is independent from the sample of women. In 2006, the NSFG transitioned from a periodically conducted survey to a continuous fieldwork design, which increased the number of interviews by 80 percent for about one-third of the cost. The continuous design has also enabled release of public-use data files every 2 years. The 2013-2015 public-use file was released in fall 2016.

Chandra described the steps involved in selecting an NSFG sample. First, primary sampling units (PSUs) are selected, consisting of Metropolitan Statistical Areas, counties or groups of counties, and then neighborhoods and secondary sampling units are selected from within those PSUs. A household screener is conducted for selected households to locate eligible men and women ages 15-49 for interview (until September 2015 the range was 15-44). The interviewers are women and are trained for a week

exclusively on the NSFG (including refusal training). The average interview length is 80 minutes for females and 60 minutes for males; the majority is interviewer administered on a laptop, but 15 to 20 minutes of each interview is self-administered using audio content (ACASI). Chandra mentioned that the latter portion contains their most sensitive content, including their questions on incarceration. Parents of NSFG respondents aged 15-17 must provide assent themselves and sign parental consent. Chandra presented a summary of the questions on criminal justice involvement in the NSFG: they are administered only to men, in ACASI mode. The questions are framed in a manner so as not to be perceived as self-incriminating. The question on “number of times” is aimed at distinguishing serious offenders from those who have committed less serious crimes:

In the last 12 months, have you spent any time in a jail, prison or a juvenile detention facility? (yes/no)

If no to the above question:

Have you ever spent time in a jail, prison or juvenile detention center? (yes/no)

If yes to either jail question on prior slide:

Have you been in jail, prison, or a juvenile detention facility only one time or more than one time? (only 1 time; more than 1 time)

If only 1 time:

How long were you in jail, prison, or juvenile detention?

If more than 1 time:

The last time you were in jail, prison, or juvenile detention, how long were you in?

Response options: One month or less; more than one month but less than one year; One year; more than one year

THE NATIONAL HEALTH AND NUTRITION EXAMINATION SURVEY

Kathryn Porter (National Center for Health Statistics) described the National Health and Nutrition Examination Survey (NHANES), which is conducted by NCHS. NHANES is a nationally representative annual survey of a sample of about 5,000 people focused on assessing the health and nutritional status of adults and children in the United States. Unlike the four surveys described above, the NHANES combines interviews and physical examinations. The data are used to produce U.S. population-based estimates of health conditions, environmental exposures, nutritional status, and diet behaviors, as well as to assess the awareness, treatment, and control of selected diseases. In the 2015 round, the survey’s response rate was

around 63 percent, which Porter noted was a disappointing decline from a decade ago in which response rates were in the high 70s.

Like the previously mentioned health surveys, Porter explained, the NHANES is administered to a sample of the civilian, non-institutionalized U.S. population. The sample is selected in four stages: counties are selected as PSUs: segments or groups of census blocks are selected within each PSU; households are selected within each segment; and participants are selected from households through screening and enumeration. Informed consent is obtained prior to both the interview and the physical examination. The consent includes language regarding the linking of NHANES data with other survey records and administrative data. Respondents are therefore agreeing not just to provide their personal data, but also to allow NCHS to link their personal data to other sources.

The home interview includes demographic, socioeconomic, dietary, and health-related questions. The examination component consists of medical, dental, and physiological measurements, as well as laboratory tests administered by trained medical personnel. The collected biological specimens are either banked or sent to laboratories for processing. Physical examination takes place in mobile examination centers that are equipped with medical devices to assess cardiovascular health, conduct exams for sensory impairments, and collect anthropometric information. Sensitive questions, such as those on mental health, reproductive health, drug use, and alcohol use, are asked of participants in the mobile examination center.

Porter informed the workshop participants about the procedure to add new content to the NHANES. The program openly invites new or revised questionnaire material, laboratory assessments, and examination contents from current and potential future collaborators from federal and state governments, the private sector, and universities. There is a 2-year timeline for incorporating suggestions, which starts with a letter of intent, followed by clearances from the U.S. Office of Management and Budget and the relevant IRB, cognitive testing of proposed questions or assessments, and training of interviewers, ending with fielding of new content.

DISCUSSION

Schober thanked the session speakers and opened the floor to discussion by requesting, first, that the presenters comment on the discussions and presentations earlier in the workshop.

Hughes talked about the tradeoffs involved in the redesign of a survey and how context effects can affect the estimates and distribution of variables. Context effects can create disruption in the general picture of how things are moving in a particular direction. In case of the NSDUH, he suggested expansion of the current criminal involvement module rather

than placing new questions in a separate section of the survey—primarily because the criminal justice module is already designed to be administered by ACASI, and the respondent is already privately answering questions.

Hughes then turned to some examples of situations in which additional questions on a survey generated inconsistent responses that rendered the data unusable. One such example was the result of a change in definition of binge drinking for women by the National Institute on Alcohol Abuse and Alcoholism from five or more drinks to four or more drinks. Instead of redesigning the alcohol module, a new question that took into account the changed definition of binge use was introduced in another module so as to not disturb trends in estimates: as a result, however, it generated inconsistent responses. The data from the new question became usable only after the questions in the alcohol module were redesigned to reflect the changed definition.

Jordyn White (Committee on National Statistics) asked when criminal justice questions were introduced to the NSDUH. Hughes answered that they were introduced in 1991 and were very basic, asking whether the respondent has ever been arrested, been on parole, or on probation. In the mid-1990s, NSDUH partnered with the Bureau of Justice Statistics, which led to the inclusion of questions on specific offenses (the list is drawn from Federal Bureau of Investigation’s Uniformed Crime Report list of offenses) to generate estimates of their prevalence. The estimation was a challenge given the sample size of the survey.

Cynamon followed up on Hughes’s answer by reminding workshop participants of the drop in response rates in household surveys. She said she is especially concerned about the fact that the demographic group with the largest nonresponse rate is young adult minority males, which affects the total number of observations from that group and in turn leads to issues of undercoverage and biased estimates.

Chandra reiterated the need to balance challenges when adding new questions to a survey. First, there are concerns that they can cause disruption in time series of variables that have been in core modules since the beginning of the survey. But by placing additional questions in a separate module to protect the content of the core module, the additional questions do not derive the benefit of the context set up in the core module. She noted that NSFG’s timeframe for making questionnaire changes is very similar to that of the NHANES: the survey is currently considering changes that would be implemented in September 2017.

Giles relayed concerns from BRFSS coordinators regarding the length of the current questionnaire, which would therefore make adding criminal justice content in BRFSS’s core module all the more difficult: it would either increase respondent burden and length or require the survey to remove a current item. He agreed with the other presenters’ concerns about the

effects of context and validity. The BRFSS’s rotating modules’ placement on the questionnaire does not vary from one year to the next—a decision that helps to protect the integrity of the core module. He added that the program will begin soliciting suggestions for content modifications and additions for the 2018 BRFSS in fall 2016, which would be the earliest opportunity to add criminal justice questions in the BRFSS.

ADDING QUESTIONS TO HEALTH SURVEYS: ADDITIONAL CONSIDERATIONS AND CONCERNS

Josiah Rich (Brown University Medical School) thanked the presenters for describing five important national health surveys. He challenged the presenters to think about three possible approaches:

1. linking their respective survey data to the National Correctional Reporting Program’s national database of prisoners and the Federal Bureau of Prisons database, which can generate data on prevalence and help validate questions regarding incarceration and health;
2. expanding the coverage of their surveys to include representative subsamples of correctional populations, which would aid in a comparison of health outcomes of the correctional populations relative to general populations; and
3. including content on opiate use disorder and overdose.

Responding to Rich’s third point, Porter described results of analysis done on prescription drug data for 10 years from the NHANES: there has been an increase in prescription opioid use, from 4 percent to 6 percent. In addition, the proportion of people taking opioids stronger than morphine has greatly increased. She noted that conducting physical examinations for people in correctional institutions would require changes in the authorizing legislation for the NHANES. An alternative would be to share NHANES methods and protocols with a private-sector contractor that could go into jails and prisons and collect biological specimens. She gave an example of the collaboration with the New York City (NYC) Department of Health and Mental Hygiene to design and implement a community HANES. The NYC HANES measured key health indicators in a randomly selected sample of adult residents through a detailed health interview and brief physical examination.

However, Porter said, she has reservations about linking NHANES survey data to administrative data on correctional and prison population because the annual NHANES sample size is only 5,000, and a very small percentage of respondents have any contact with the criminal justice system. She opined that aggregating 10 years of survey data may produce a

sufficient sample size that can be linked with databases—one of the options Rich suggested—but for such an exercise to bear fruit it would require formulation of research questions and investigation of statistical power of the survey to provide answers to those questions.

Regarding the NSDUH, Hughes said that when the survey expanded the prescription drug module significantly by adding more drugs, the usage data broke prior trends. While there had been a general decline in prescription opioid use, the frequency of usage among heavy opioid users has been increasing. In responding to Porter’s point that survey mandates do not cover correctional populations, Hughes said he was not sure if there is added value in linking NSDUH data to databases on correctional and prison populations.