Psychological Testing in the Service of Disability Determination (2015)

2

Disability Evaluation and the Use of Psychological Tests

In 2013, the U.S. Social Security Administration (SSA) received approximately 2.6 million applications for Social Security Disability Insurance (SSDI) disabled worker benefits (SSA, n.d.-m), 1.6 million applications for the Supplemental Security Income (SSI) adult program (SSA, 2014a, p. 92, Table V.C.1), and 442,000 applications for the SSI child program (SSA, 2014a, p. 24, Table V.C.2). This chapter describes SSA’s process for evaluating applications and determining the disability status of the applicants, including the use of psychological testing in SSA disability evaluations. It also provides an overview of base rates of “malingering” and a discussion of the benefits of formal, standardized data collection and actuarial data interpretation. The chapter concludes with an overview of the use of psychological tests in disability evaluations in non-SSA systems, including the U.S. military and the U.S. Department of Veterans Affairs (VA), private disability insurance, forensic assessments, and some international programs.

SOCIAL SECURITY ADMINISTRATION DISABILITY DETERMINATION PROCESS

The overall disability determination process (see Figure 2-1) is the same for both SSDI and SSI, although the specific steps of the process vary for adults (20 CFR § 416.920; see Figure 2-2) and children (20 CFR § 416.924; see Figure 2-3). For the average applicant, the initial determination process takes between 90 and 120 days from the date of filing. Decisions for applicants with certain medical conditions, incomplete medical records, or

FIGURE 2-1 Overview of the SSA disability process.

FIGURE 2-2 Disability determination process for adults by the numbers.

SOURCES: SSA, 2014d,h.

SOURCES: SSA, 2014d,h.

FIGURE 2-3 Disability determination process for children by the numbers.

SOURCE: SSA, 2014h.

who appeal the initial decision can take far longer, in some cases stretching across several years (SSA, 2014i; SSDRC, n.d.).

Step 1: Nonmedical Eligibility?

Applications for disability benefits are made at a local SSA field office. During the first step of the disability determination process, officials in the SSA field offices verify applicants’ financial and other nonmedical (e.g., age, work credits) eligibility requirements (SSA, 2012a). For SSDI and SSI applicants, the examiners first check to see if applicants are currently working and earning more than the substantial gainful activity (SGA) amount—$1,040 per month in 2013 for non-blind applicants (SSA, 2014m). For SSI applicants, examiners also verify that applicants meet the income and resource limits necessary to qualify for these means-tested benefits.¹ For concurrent SSDI/SSI adult applicants, financially eligibility is checked for both programs. If applicants fail on any of these financial criteria, the application is denied.

If an applicant meets the nonmedical eligibility requirements, the application is forwarded to the state Disability Determination Services (DDS)

_____________

¹ For SSI child applicants, the income test relates to the resources of the household.

agency, where a disability examiner develops and reviews the medical and other evidence² for the claim and makes an initial determination about disability. In 2013, state DDS offices evaluated approximately 2.8 million applications for disability benefits distributed as follows: 915,679 SSDI; 887,506 concurrent SSDI/SSI adult; 653,699 SSI adult; and 428,208 SSI child (SSA, 2014h). Before beginning the disability evaluation, DDS examiners recheck that applicants meet the financial and other nonmedical criteria for the disability programs. As shown in Figure 2-2, almost no cases that reach the DDSs are rejected at this step, because the SSA field offices have already screened the applicants on these criteria. If the financial criteria are met, the DDS agencies begin to develop the case.

DDS agencies follow either a traditional or a single-decision-maker (SDM) model (see Figure 2-1), depending on the state. In the traditional model, the disability examiner makes the determination in conjunction with a DDS psychological consultant or a medical consultant (20 CFR § 404.1615). In the SDM model (20 CFR § 404.906), disability examiners have the authority to make the initial disability determination. In most cases, the disability examiners prepare the assessments and have the authority to approve or deny claims without obtaining the signature of a medical or psychological consultant. The exception is denials for mental impairments, which must be reviewed by a psychological consultant. Medical and psychological consultants are always available to assist disability examiners in their review of claims.

Step 2: Severe Impairment?

The second step of the process is designed to screen out applicants whose medically determinable impairments are not considered to be “severe”—i.e., those who are clearly able to work at some sort of substantial gainful activity or whose impairment is expected to resolve within 12 months. A medically determinable physical or mental impairment or combination of impairments is considered severe “if it significantly limits an individual’s physical or mental abilities to do basic work activities” (SSA, 1996a). The impairment also must either be expected to result in death or have lasted (or be expected to last) for 12 continuous months. An applicant is denied at this step if the medically determinable impairment or combination of impairments “has no more than a minimal effect on the ability to do basic work activities” (SSA, 1996a) or does not meet the duration criterion. In 2013, 9.5 percent of SSDI applicants, 17.8 percent of SSDI/SSI concurrent

_____________

² Types of evidence may include (1) objective medical evidence—i.e., medical signs and laboratory findings, (2) medical history and treatment records, (3) medical source opinions and statements, (4) statements from claimant or others, and (5) information from other sources—e.g., educational personnel, social welfare agency personnel (SSA, 2012b).

applicants, and 7.0 percent of SSI adult applicants were denied at this step (see Figure 2-2) (SSA, 2014h). If the applicant is found to have a severe impairment, the disability evaluation moves to the next step.

Step 3: Meets or Equals Medical Listings?

At Step 3, applicants’ impairments are evaluated to determine whether they meet or equal the medical criteria codified in SSA’s Listing of Impairments for adults (SSA, n.d.-c). The Listing of Impairments is organized by major body system and contains criteria to evaluate the severity of a listed impairment. These criteria may include assessments of work-related functioning³ and are designed to identify individuals with impairments that are sufficiently severe to prohibit them from engaging in any kind of “gainful activity” (SSA, n.d.-b). In some cases, an individual has multiple impairments, none of which is, by itself, sufficiently severe to meet the listing criteria, or an impairment that is not included in the Listing. In such cases, the examiner considers whether the impairment or combination of impairments is medically equal to a listed impairment. If an applicant’s impairment(s) meets or equals the listing criteria, the claim is allowed. In 2013, 17.8 percent of SSDI applicants, 11.2 percent of SSDI/SSI concurrent applications, and 14.1 percent of SSI adult applicants were allowed at this step of the disability screening process (see Figure 2-2) (SSA, 2014h). All remaining claims move to the fourth step in the evaluation process.

Step 4: Capacity for Past Work?

At this step, applicants are assessed with respect to their mental or physical “residual functional capacity” and the extent to which they can still perform activities related to jobs they have held in the past 15 years. Applicants who are found to meet the demands of “past relevant work” are denied. In 2013, 14.1 percent of SSDI applicants, 11.7 percent of SSDI/SSI concurrent applicants, and 5.9 percent of SSI adult applicants were denied at this step of the process (see Figure 2-2) (SSA, 2014h). Applicants who no longer are able to perform work they have done in the past are then assessed for their ability to perform any work in the national economy (Step 5).

_____________

³ For mental disorders, functional limitations are used to assess the severity of the impairment. Paragraph B and C criteria in the Listing of Impairments for mental disorders describe the areas of function that are considered necessary for work (SSA, 2009).

Step 5: Capacity for Any Work?

At this step, applicants’ residual functional capacity is evaluated along with the vocational factors of age, education, and previous work experience to determine whether they would be able to adjust to other work that exists in the national economy. Disability examiners consider increasing age, generally beginning at age 50; years of education or specialized job or vocational training; and transferability of skills from previous employment, along with an individual’s residual physical and mental abilities, when determining whether the applicant could adjust to doing some sort of work (SSA, n.d.-j). For example, a 50-year-old applicant with less than a high school education, no skilled work experience, and a maximum sustained work capacity limited to sedentary work could be considered disabled, while the same 50-year-old applicant who has experience as a skilled worker could be denied. If an applicant is found unable to perform any work in the national economy, the claim is allowed; otherwise, the claim is denied. In 2013, 24.3 percent of SSDI applicants were denied benefits at this stage, and 25.5 percent were determined to be eligible for benefits (see Figure 2-2) (SSA, 2014h). Among SSDI/SSI concurrent applicants, 33.8 percent were denied at Step 5, and 12.5 percent were allowed (see Figure 2-2) (SSA, 2014h). Among SSI adult applicants, 40.1 percent were denied at Step 5, and 13.9 percent were allowed (see Figure 2-2) (SSA, 2014h). Notably, more than 50 percent of the initial determinations made at the DDS level in 2013 were made in this final step of the disability determination process, when medical-vocational factors are a primary component of the determination decision.⁴

SSA is in the process of updating its system for making medical-vocational decisions (SSA, n.d.-l). The medical-vocational decisions require up-to-date information about the occupations that exist in the national economy. Through an interagency agreement with the U.S. Bureau of Labor Statistics (BLS), SSA is working to develop an Occupational Information System (OIS). The OIS would include data elements of interest to SSA, including data elements that describe the mental and cognitive demands of work, on the full range of occupations available in the national economy.

At the end of the five-step determination process, 43.7 percent of SSDI applicants, 23.8 percent of SSDI/SSI adult concurrent applicants, and 28.1 percent of SSI adult applicants in 2013 were awarded benefits during the initial determination process (SSA, 2014h).⁵ As described below, applicants

_____________

⁴ The large number of cases determined on medical-vocational criteria is not unusual or unique to 2013.

⁵ These figures are obtained by summing the percentages shown in Figure 2-2 for denied and allowed applicants across all stages. Applications for SSDI and SSI adult benefits may be initially denied at any point along the five-step determination process. Applications may be allowed only at Steps 3 and 5.

denied benefits during this initial evaluation process may be eligible for appeal. As such, the allowance rates from this initial evaluation stage are lower than the final allowance rates for all applicants.

Sequential Disability Determination Process for Children

The first two steps of the disability determination process are similar for children younger than 18 years of age and adults. As with SSDI and SSI adult applications, almost no applications are rejected at Step 1 due to prescreening of the nonmedical eligibility requirements by the SSA field offices. Step 2 for children involves a determination of whether the child has a medically determinable impairment or combination of impairments that causes more than “minimal functional limitations” rather than whether it precludes substantial gainful activity as in the adult cases (20 CFR § 416.924). In 2013, 6.1 percent of SSI child applications were denied at Step 2 (see Figure 2-3) (SSA, 2014h). As with adults, Step 3 involves a determination of whether a child’s medically determinable physical or mental impairment(s) meets or medically equals the clinical criteria in SSA’s Listing of Impairments for children (SSA, n.d.-d). If so, the claim is allowed. In 2013, 19 percent of SSI child applications were allowed at this stage (see Figure 2-3) (SSA, 2014h).

The primary difference between disability evaluations for children and adults is in an additional component of the evaluation at Step 3 for children whose impairments do not meet or medically equal the listings. In these cases, the examiner considers whether the impairment results in limitations that functionally equal the medical listings (20 CFR § 416.926a). To be functionally equal to the listings, the impairment must result in “marked” limitations in two of six domains of functioning or an “extreme” limitation in one of the domains.⁶ The six domains considered are “(1) acquiring and using information, (2) attending and completing tasks, (3) interacting and relating with others, (4) moving about and manipulating objects, (5) caring for oneself, and (6) health and physical well-being” (20 CFR § 416.926a). In making the assessment, the examiner considers all of the information in the record about the interactive and cumulative effects of the impairments, including any that are not “severe,” on the child’s functioning during all activities at home, at school, and in the community. The assessment is based on how “appropriately, effectively, and independently” the child performs these activities compared to children of the same age who do not have

_____________

⁶ A limitation is “marked” if it seriously interferes with the child’s ability to independently initiate, sustain, or complete activities and is “extreme” if it very seriously interferes with the child’s ability to independently initiate, sustain, or complete age-appropriate activities (20 CFR § 416.926a).

impairments (20 CFR § 416.926a). If the child’s impairment functionally equals the severity of the medical listings, the application is approved. In 2013, 21.1 percent of applications were allowed and 48.6 percent were denied at this final step (see Figure 2-3) (SSA, 2014h).

The remaining steps of the disability determination process for adults, Steps 4 and 5, do not pertain to children. Summing the allowances in at Steps 2 and 3 (see Figure 2-3) brings total allowances in the initial determination stage to 40.1 percent (SSA, 2014h).The remaining cases were denied during the initial determination process. As with adults, denied applicants are allowed to appeal their decision, potentially increasing the final allowance rate for the program.

Medical and Other Evidence and Consultative Exams

The DDS uses the medical and other evidence in the applicants’ files in making disability determinations. SSA recognizes different categories of evidence, including (1) objective medical evidence; (2) narrative medical records, opinions, and statements from treating and nontreating medical sources; (3) statements by the applicant for the file or made to medical sources or SSA field office or DDS representatives; and (4) information from other nonmedical sources (e.g., educational personnel, social welfare agency personnel). More generally the categories can be grouped as “objective medical evidence,” applicant self-reports, and third-party reports (medical and nonmedical). According to SSA regulations, objective medical evidence refers to medical signs⁷ and laboratory findings.⁸ Laboratory findings must be demonstrated through “medically acceptable laboratory diagnostic techniques,” among which SSA includes psychological tests (20 CFR § 404.1528).

SSA’s use of the term objective medical evidence to refer to observable medical signs and laboratory or test results implies that the other types of evidence are “subjective” and therefore, perhaps, less reliable, which creates a tension among the different types of evidence that SSA considers.

_____________

⁷ “Signs are anatomical, physiological, or psychological abnormalities which can be observed, apart from [self-reported symptoms]. Signs must be shown by medically acceptable clinical diagnostic techniques. Psychiatric signs are medically demonstrable phenomena that indicate specific psychological abnormalities, e.g., abnormalities of behavior, mood, thought, memory, orientation, development, or perception. They must also be shown by observable facts that can be medically described and evaluated” (20 CFR § 404.1528).

⁸ “Laboratory findings are anatomical, physiological, or psychological phenomena which can be shown by the use of medically acceptable laboratory diagnostic techniques. Some of these diagnostic techniques include chemical tests, electrophysiological studies (electrocardiogram, electroencephalogram, etc.), roentgenological studies (X-rays), and psychological tests” (20 CFR § 404.1528).

This may arise particularly for categories of claims in which impairments are established and assessed primarily on reports of signs and symptoms of impairment and functional limitation (e.g., mental impairments other than intellectual disability, certain musculoskeletal conditions). It is important to note, as discussed in Chapter 4, that self-report measures can be valid assessment tools. In addition, SSA considers the consistency of all the evidence in a record to establish confidence in the validity of the claim of impairment and functional limitation.

If the information is insufficient to make a determination, the examiner generally tries to obtain additional information from the applicant’s medical sources and, in some cases, other sources. Medical reports should include the applicant’s medical history, clinical and laboratory findings, diagnosis, and prescribed treatment, including the applicant’s response and prognosis. In addition, the report should include a statement about what the applicant can still do, including, for adults, the physical and/or cognitive ability to perform work-related activities. For children, the statement should discuss the child’s functional limitations relative to other children of the same age (SSA, n.d.-a).

If the information requested from the applicant’s treating and other sources is unavailable or remains insufficient (e.g., lacking in necessary detail or conflicting, inconsistent, or ambiguous) to make a determination, DDS may arrange for a consultative examination (CE) to obtain additional information needed to evaluate the claim (20 CFR § 404.1519a). In 2013, 45.1 percent of disability applicants received a CE as part of the initial disability determination process (SSA, 2014d). CEs were more commonly acquired for SSI and concurrent SSDI/SSI adult applicants than for SSDI applicants (SSA, 2014d). The minimum requirements for CE reports for mental disorders in adults and children can be found in the SSA’s consultative examination guide for health professionals (SSA, n.d.-k). (See also for adults, SSA [2014e] and for children SSA [2012c].)

Appeals Process

If the DDS denies an application, the applicant can appeal the decision in turn to (1) the DDS (reconsideration), (2) an administrative law judge (ALJ), (3) the Appeals Council, and (4) a federal court.⁹ Data on the number of applicants who appeal their decision at each stage are available from SSA. Because it takes time for denied applicants to move through the various stages of the appeal process, data are available through 2010. The data show that approximately 55 percent of those who applied for

_____________

⁹ A 10-state pilot program begun in 1999 permits a claimant to bypass reconsideration by DDS and submit the appeal directly to an ALJ.

SSDI or concurrent worker benefits in 2010 and were denied during the initial evaluation, appealed the decision (calculation based on data from the 2013 Annual Statistical Report on the SSDI program, Tables 61 and 62 [SSA, 2014b]).¹⁰ The rates of appeal were slightly lower for denied SSI applicants. Approximately 45 percent of 2010 SSI adult applicants and 30 percent of 2010 SSI child applicants who were rejected in the initial determination process appealed their decisions (calculations based on data from the 2013 Annual Statistical Report on the SSI program, Tables 70 and 71 [SSA, 2014k]).

The first level of appeal, which takes place within the DDS, is a reconsideration of the original claim or, for SSI, a review of an initial determination. Reconsideration involves a complete review of the initial claim by an examiner and, where applicable, a medical consultant who did not participate in the original evaluation. DDSs are reported to approve about 5 percent of reconsideration claims (Morton, 2014).

If the reconsideration is denied, the next level of appeal is a hearing before an ALJ. ALJs are employed by SSA and, on appeal, review the evidence in an applicant’s file, including any new evidence submitted by the applicant. The ALJ also may interview the applicant and any witnesses brought by the applicant, as well as relevant medical or psychological consultants, other health care providers, or vocational experts. The applicant or a representative also may question any of the other witnesses. After considering all of the evidence and testimony, the ALJ issues a written decision (SSA, n.d.-i). If the ALJ finds that additional evidence is needed, he or she may order a CE or otherwise seek further development of the case file (SSA, 2012f). Reportedly about 67 percent of the claims reviewed by ALJs overall are approved, although the approval rate varies among ALJs and can be much higher (Morton, 2014; SSA, 2015).

Claims that are denied at the ALJ level may be brought to the Appeals Council, which serves as the final level of appeal within SSA. The Appeals Council considers each case brought to it and either denies the request for review, if it agrees with the ALJ’s decision; sends it for review by another ALJ, if it finds a technical or procedural error with the ALJ’s decision; or decides the case itself and grants benefits to the applicant (Laurence, 2015; SSA, n.d.-h). About 16 percent of requests for review are returned for re-review by an ALJ. In fiscal year 2014, the Appeals Council received more than 155,000 new requests for review. The council processed more than 162,280 requests that year. The processing time averaged 374 days.¹¹

_____________

¹⁰ This figure includes concurrent SSDI/SSI applicants.

¹¹ The numbers in this paragraph have been updated from those provided in the prepublication version of the report and were provided by SSA on May 13, 2015.

If the Appeals Council dismisses or does not reverse an unfavorable decision by the ALJ, the applicant may contest SSA’s final decision by filing a civil suit in U.S. district court (SSA, n.d.-g). In fiscal year 2013, more than 18,700 new cases were filed (SSA, n.d.-g). The federal judge agrees with or overturns the decision of the ALJ and the Appeals Council, thereby denying or awarding benefits, or sends the case back for re-review by the ALJ.

Returning to data for 2010, by the end of all stages of the appeal process, 53 percent of SSDI or concurrent worker applicants who appealed their initial denial ultimately received an award (calculation based on data from the 2013 Annual Statistical Report on the SSDI program, Tables 62 and 63 [SSA, 2014b]). The rates are lower for SSI applicants: 40 percent of SSI adult applicants and 27 percent of child applicants in 2010 were ultimately awarded benefits after appeal (calculations based on data from the 2013 Annual Statistical Report on the SSI program, Tables 71 and 72 [SSA, 2014k]).

Final Outcomes of the Disability Determination Process

The final award rate, which includes initial and appealed decisions, varies across disability programs but is always higher than the initial award rates given in Figures 2-2 and 2-3. Based on data for applicants who filed for benefits in 2010, final award rates for disability benefit applicants are around 55 percent for SSDI workers, including concurrent applicants; 40 percent for SSI adult applicants; and 45 percent for SSI child applicants (SSA, 2014b, Tables 61, 62, 63, 2014k, Tables 70, 71, 72).¹²

Variability in Outcomes Across States

Although state DDS offices and SSA follow the same disability determination and appeals process, award rates vary significantly by state, reflecting variation in both filing rates (applications per eligible population) (see Figure 2-4) and allowance rates (allowances per DDS determinations) (see Figure 2-5). Variation in these rates stems, in part, from factors outside of the direct control of DDS offices or SSA. Such factors include state-level differences in population characteristics, such as age, education, and impairment type, as well as differences in local labor market

_____________

¹² In 2010, there were still applications pending final approval. Allowance rates for earlier years with smaller numbers of pending decisions were slightly higher than those referenced here for 2010.

FIGURE 2-4 Filing rates by state, fiscal year 2013.

SOURCES: SSA, 2014b,k.

FIGURE 2-5 Allowance rates by state, fiscal year 2013.

SOURCES: SSA, 2014b,k.

conditions, such as the unemployment rate or mix of jobs available for workers with different skills.¹³

Several studies have attempted to quantify the degree to which state variation in application, allowance, and award rates is explained by these factors. In general the results suggest that observable state and individual characteristics account for half or more of the total variation. For example, Strand (2002) finds that controlling for state-level observables and year effects reduced variation in state-level allowance rates (1997–1999) by half. Soss and Keiser (2006) find similar reductions in variation for SSDI and SSI application rates.

Rupp (2012) decomposes overall cross-state variation in allowance rates for the 1993–2008 period and attributes it to one of four sources: (1) time-varying independent variables (unemployment rate and demographic and diagnostic criteria); (2) year fixed effects that capture national changes in economic conditions or policies affecting disability programs; (3) state fixed effects that capture unobservable, long-term differences across states that may or may not be related to DDS management; and (4) residual unexplained that captures the remaining variation not associated with any of the model variables (see Table 2-1).

The results show that time-varying independent variables explain a relatively small share of the state variation in allowance rates; about 10 percent for SSDI allowance rates and about 20 percent of variation in adult SSI and concurrent SSDI/SSI claims. Only 6 percent of the total variation in SSI child allowance rates is accounted for by the time-varying independent variables included in his model. Year fixed effects account for an additional small share of the variation in adult allowance rates (SSDI and SSI) but nearly 30 percent of the variation in SSI child allowances. Notably, between 40 and 50 percent of the overall variation in allowance rates across states is explained by long-term, unobservable state-specific differences. Combining these numbers with the amount unexplained by the model, the total variation in state allowance rates that cannot be traced back to observable variables outside of the DDS control is approximately 75 percent.

Although it is not possible to know definitively whether the large share of unexplained variation in state filing, award, and allowance rates is driven

_____________

¹³ A long literature has documented the relationship between local labor market conditions, generally measured by the unemployment rate, and applications and awards for disability benefits. In general the results show that poor economic conditions/higher unemployment rates are associated with increased applications and awards for benefits (Autor and Duggan, 2003; Black et al., 2002; Burkhauser et al., 2002; Duggan and Imberman, 2008; Kreider, 1999; Rupp and Stapleton, 1995). Research on allowance rates and economic conditions (Rupp, 2012; Rupp and Stapleton, 1995; Strand, 2002) generally finds a negative relationship suggesting that SSA is able to screen out some marginally qualified candidates who might apply for the program in response to poor economic conditions.

TABLE 2-1 Components of Total Variation in Allowance Rates from Level Fixed-Effects OLS Regression Models, by SSA Program Group (in percent), 1993–2008

NOTES: A total of 12 regressions were estimated: 3 models for each of the 4 program groups. For each program group, independent variables were included in a sequential manner. The first model included only state fixed effects. The second model added year fixed effects. The third model added the time-varying variables. The results in this table reflect state-level OLS regression models. Totals may not sum to 100 because of rounding.

^a The first row contains the R² from the first model for each program group. The subsequent two rows reflect the marginal increase in the R² arising from adding the given group of independent variables to the model. The total of the first three rows represents the R² for the third model that included all three groups of variables.

^b The unexplained variation was calculated by subtracting the R² for the third model that included all of the predictors from 100 percent.

SOURCES: Data are based on 1,736,554 initial disability determinations in the 50 states and the District of Columbia for the 1993–2008 period, taken from SSA’s National Disability Determination Services System File. State unemployment rate data are taken from the Current Population Survey. Reprinted with permission from Rupp, 2012, Table 9.

by variability in the federal disability determination process, there is some evidence that states differ in how they manage claims. For example, there are significant differences across states in the percentage of cases requiring a CE as part of the initial determination. Recall that nationally about 45 percent of initial determinations request a CE. By contrast, in low-CE states such as Hawaii, Missouri, and Virginia about one-quarter of cases receive a CE (SSA, 2014c). In high-CE states such as Indiana, Kentucky, and Tennessee about two-thirds of initial determinations request a CE (SSA, 2014c). That said, because the committee could locate no study of the variability of CE rates, this evidence is only suggestive of differences in case management across states.

COMPOSITION OF SSA BENEFICIARIES

Although there are no data on the composition of impairments affecting applicants, the data on allowed claims provide insight into the types of individuals seen at the state DDS offices. Figure 2-6 shows the composition of new beneficiaries in 2013 for SSDI and SSI adults and children. By far the largest two impairment categories for all three disability programs are mental disorders (excluding intellectual disabilities) and musculoskeletal and connective tissue disorders. In 2013, these two categories accounted for 52 percent of new SSDI awards, 53 percent of new SSI adult awards, and 58 percent of new SSI child awards. Within these two categories, a significant fraction of the applicants have conditions, including affective mood disorders and disorders of the back, for which the presence and severity of impairment and associated functional limitations are based largely on applicant self-report (SSA, 2014j,l).

The large share of these two categories in the flow of new beneficiaries indicates that DDS offices are evaluating a large number of cases that require more subjective judgment about the functional limitations the client faces. This is supported by the large number of adult cases that are determined on medical-vocational criteria at Steps 4 and 5 of the determination process: more than 50 percent of the initial DDS decisions and more than 80 percent of decisions at the hearing level (SSA, n.d.-l).

PSYCHOLOGICAL TESTING IN SSA DISABILITY EVALUATIONS

Policy Relevant to Evaluations of Disability for Mental Disorders

Adults who file for SSA disability on the basis of mental disorders and meet the nonmedical eligibility criteria are evaluated at Step 2 for the presence of a medically determinable mental impairment, the severity of the functional limitation it imposes on the individual’s ability to work, and a determination that the impairment has lasted or will last for 12 or more continuous months (SSA, 2012d, n.d.-e). The DDS assesses the presence of a medically determinable mental impairment on the basis of the medical evidence, including relevant signs, symptoms, and laboratory or psychological test findings (SSA, 2012d).

The DDS assesses the severity of a medically determinable mental impairment on the basis of the functional limitations it imposes on the claimant’s ability to engage in work-related activities. Functional limitations are assessed in four areas that are considered essential for work: (1) activities of daily living (ADLs); (2) social functioning; (3) concentration, persistence, or pace; and (4) episodes of decompensation in a work-like setting—or “the

FIGURE 2-6 Composition of new beneficiaries in 2013 for SSDI and SSI adults and children.

SOURCES: SSA, 2014b,k.

ability to tolerate increased mental demands associated with competitive work” (SSA, 2009, section B). These areas correspond to the Paragraph B criteria,¹⁴ which are part of the listings of impairments for mental disorders assessed at Step 3. A functional limitation is considered “marked” if it is “more than moderate but less than extreme”; in other words, the degree of limitation “interfere[s] seriously with [the claimant’s] ability to function independently, appropriately, effectively, and on a sustained basis” (SSA, n.d.-e, section C).

ADLs and social functioning are evaluated within the contexts of (1) appropriateness, (2) independence, (3) sustainability, (4) quality, and (5) effectiveness (SSA, 2009). Information about the claimant’s ADLs

_____________

¹⁴ Under a notice of proposed rulemaking, SSA has proposed revised Paragraph B criteria to capture “the mental abilities an adult uses to function in a work setting” (SSA, 2010, p. 51340). The revised B criteria are the abilities to “understand, remember, and apply information”; “interact with others”; “concentrate, persist, and maintain pace”; and “manage oneself.”

and social functioning is acquired through interview, self-report, observation, and other report. Concentration, persistence, or pace “refers to the ability to sustain focused attention sufficiently long to permit the timely completion of tasks commonly found in work settings” (SSA, 2009, section D). These functions may be assessed with a mental status exam or psychological tests, but such tests represent a point in time and do not necessarily reflect the ongoing stresses of a work environment. Clinical and test data should be supplemented by other evidence, such as observations of performance in a work or work-like setting.

The inability to tolerate the increased demands associated with work (deterioration or decompensation) is demonstrated by an increase in the signs or symptoms and the need for new or additional treatment or removal from the stressful environment. Generally to meet the criteria the claimant would have had at least three episodes, each lasting 2 weeks or longer, in the most recent year.

Step 2 is the first point at which the results of cognitive and non-cognitive tests can help inform SSA’s disability determination process. The results of such tests can help support the identification and documentation of the presence and severity of medically determinable mental impairments. It is important to note that an individual’s level of functioning can fluctuate over time. To evaluate an individual’s impairment accurately, it is important for DDS examiners to obtain evidence across a long enough timeframe (SSA, 2012d).

Applicants who meet the criteria at Step 2 are evaluated at Step 3 to determine whether they meet or equal the criteria in the Listing of Impairments for mental disorders (SSA, n.d.-e, n.d.-f). The listings for mental disorders include 9 diagnostic categories for adults¹⁵ and 11 categories for children, of which the first 9 are similar to the adult listings:

1. Organic mental disorders
2. Schizophrenic, paranoid, and other psychotic disorders
3. Affective (mood) disorders
4. Intellectual disability disorders
5. Anxiety-related disorders
6. Somatoform disorders¹⁶
7. Personality disorders
8. Substance addiction disorders
9. Autistic disorder and other pervasive developmental disorders
10. Attention deficit hyperactivity disorder (children)

_____________

¹⁵ Under the same notice of proposed rulemaking (SSA, 2010), SSA has proposed revised listing categories.

¹⁶ Somatoform disorders are discussed separately in the following section.

11. Developmental and emotional disorders of newborn and younger infants (children)

For most of the diagnostic categories,¹⁷ adult applicants will meet a listing if the impairment satisfies the following: (1) the diagnostic description of the mental disorder; (2) specified medical findings—e.g., symptoms (self-report), signs (medically demonstrable), laboratory findings (including psychological test findings)—(Paragraph A criteria); and (3) specified “impairment-related functional limitations that are incompatible with the ability to do any gainful activity” (Paragraph B or Paragraph C criteria) (SSA, n.d.-e). Paragraph A criteria, in conjunction with the diagnostic description, substantiate the presence of the specific mental disorder based on the medical evidence. Paragraph B and Paragraph C criteria list the functional limitations resulting from the mental impairment that preclude the ability to engage in gainful activity. Cognitive and non-cognitive test results can inform disability determinations at Step 3, particularly with respect to Paragraph A and B criteria.

If an applicant’s impairment does not meet the diagnostic definition or the Paragraph A criteria of a listing but does result in the functional limitations specified in the Paragraph B or C criteria, the impairment is considered to equal the listing. Applicants whose impairments are severe but do not meet or equal any of the listings are not approved at Step 3. They move on to an evaluation of their residual function capacity at Steps 4 and 5 of the determination process. Residual functional capacity refers to the work-related capacities an applicant still possesses despite the impairment. Assessment of residual functional capacity is another area of the determination process that the results of psychological testing could inform.

The determination process differs somewhat for children at Step 3. In addition to asking whether the child’s impairment(s) meets or medically equals one of the listings, a second question is posed if it does not: Does the impairment functionally equal the listings? By “functionally equal the listings,” SSA means that “the impairment(s) must be of listing-level severity; i.e., it must result in ‘marked’ limitations in two domains of functioning or an ‘extreme’ limitation in one domain” (20 CFR § 416.926a). The functional limitations caused by the child’s impairment(s) are assessed. In determining functional equivalence, SSA considers “the interactive and cumulative effects of all of the impairments for which [it has] evidence,

_____________

¹⁷ The structure of the listing for intellectual disability and for substance addiction disorders differ from that of the other mental disorder listings. There are four sets of criteria (Paragraphs A through D) for the intellectual disability listing, and the listing for substance addiction disorders refers to which of the other listings should be used to evaluate the various physical or behavioral changes related to the disorder.

including any impairments [the child has] that are not ‘severe’ (see § 416.924(c))” (20 CFR § 416.926a). When assessing a child’s functional limitations, it considers “how appropriately, effectively, and independently [the child] performs … activities compared to the performance of other children [the same] age who do not have impairments” (20 CFR § 416.926a).

Documentation

As previously described, the DDS uses all relevant evidence in an applicant’s file in making a disability determination. The medical evidence in an applicant’s file must be sufficiently complete and detailed to allow the DDS to make a determination. Medical evidence includes a history of the individual’s mental impairment, the results of any mental status examinations and psychological tests, and the records of any treatments and hospitalizations provided by an “acceptable medical source” (SSA, 2014f, n.d.-e).

Although a full mental status exam, performed during a clinical interview, can be tailored to target the specific areas most relevant to the alleged impairment, a comprehensive exam generally would include “a narrative description of [the individual’s] appearance, behavior, and speech; thought process (e.g., loosening of associations); thought content (e.g., delusions); perceptual abnormalities (e.g., hallucinations); mood and affect; sensorium and cognition (orientation, recall, concentration, intelligence); and judgment and insight” (SSA, n.d.-e, section D4).

Psychological Testing

SSA understands “standardized psychological tests” to be psychological test measures that have “appropriate validity, reliability, and norms” representative of relevant populations (SSA, n.d.-e, section D5). SSA characterizes a “good test” as one that is valid (“measures what it is supposed to measure”) and reliable (use of the same test in the same individual yields consistent results over time) and has “appropriate normative data” and a “wide scope of measurement” (measures a broad range of elements of the domain being assessed) (SSA, n.d.-e, section D5).

SSA specifies the tests would be administered, scored, and interpreted by a “qualified” specialist—meaning someone “currently licensed or certified in the state to administer, score, and interpret psychological tests” with the “training and experience to perform the test” (SSA, n.d.-e, section D5). The types of specialists who are qualified to administer, score, and interpret standardized psychological tests are discussed in Chapters 3, 4, and 5. Observations of the test administrator—such as ability to concentrate, interact appropriately with test administrator, perform independently—would supplement the report of test results. The report would also address

the validity of the test results, including discussion of any discrepancies between the test results and “the individual’s customary behavior and daily activities” (SSA, n.d.-e, section D5).

The results of standardized intelligence tests are built into the listings for intellectual disability and some neurological impairments. In addition, SSA notes that intelligence test results can help to confirm the presence of intellectual disability and organic mental disorders as well as the severity of cognitive impairment. SSA states that standardized personality measures (e.g., Minnesota Multiphasic Personality Inventory-2) or projective testing techniques (e.g., Rorschach) may provide useful data for the evaluation of disability “when corroborated by other evidence, including results from other psychological tests and information obtained in the course of the clinical evaluation” (SSA, n.d.-e, section D7). SSA also states that “comprehensive neuropsychological examinations may be used to establish the existence and extent of brain function, particularly in cases involving organic mental disorders” (SSA, n.d.-e, sections D6, D7, D8).

Psychological Consultative Examinations

SSA specifies the minimum content requirements for CE reports for adults with mental disorders (SSA, n.d.-k, Part IV, Mental Disorders). These requirements include the following: applicants’ longitudinal, current, and past medical history; current medications; social and family history; physical examination; and mental status evaluation.¹⁸ In addition, the report is to include interpretation of any psychological and/or clinical test results in relation to the history and examination findings as well as identification of the individual providing the interpretation if different from the provider signing the CE report (SSA, n.d.-k, Part IV, Mental Disorders, section H). The report also is to specify “a full multiaxial classification as set forth in the current Diagnostic and Statistical Manual of Mental Disorders” and prognosis and recommendations for treatment, if indicated (SSA, n.d.-k, Part IV, Mental Disorders, section I).

For applicants with intellectual impairments, current documentation of intelligence quotient (IQ) is required along with interpretation of the results, including an assessment of their validity, and consistency of the results “with the claimant’s educational, vocational, and social background” (SSA, n.d.-k, Part IV, Mental Disorders, section I). Also required is “a

_____________

¹⁸ Elements include “(1) manner and approach to evaluation; (2) dress, grooming, hygiene and presentation; (3) mood and affect; (4) eye contact; (5) expressive/receptive language; (6) recall/memory, including working, recent, and remote; (7) orientation in all four spheres; (8) concentration and attention; (9) thought processes and content; (10) perceptual abnormalities; (11) suicidal/homicidal ideation; (12) judgment/insight; and (13) estimated level of intelligence” (SSA, n.d.-k, Part IV, Mental Disorders, section G).

comprehensive and detailed description of adaptive behavior in the areas of personal, social, academic, and occupational functioning during the developmental period” (SSA, n.d.-k, Part IV, Mental Disorders, section I).

Additionally, SSA specifies that CE reports for mental disorders should include statements from the medical source regarding “the nature and extent of the mental disorder” and “an assessment of the claimant’s abilities and limitations based on medical history, observations during examination, and results of relevant laboratory tests” as well as an opinion regarding the applicant’s ability to carry out certain functions (SSA, n.d.-k, Part IV, Mental Disorders, section J). The report should discuss “any apparent discrepancies in medical history or in examination findings and how the discrepancies were resolved”; include “a statement regarding malingering, if applicable”; and “a statement regarding the [applicant’s] capability to manage funds” (SSA, n.d.-k, Part IV, Mental Disorders, section J).

In practice, CEs for mental disorders generally consist of nonstandardized diagnostic interviews and mental status exams, with little to no standardized psychological testing other than intelligence testing (Chafetz, 2008; Chafetz et al., 2007; Griffin et al., 1996; Heiser, 2014; McLaren, 2014; Price, 2014; Ward, 2014).

Aside from the use of intelligence tests as described in the listings for intellectual disability and certain neurological impairments, SSA does not require or specify the purchase of any type of or individual psychological test. The primary guidance provided by SSA is that good psychological tests are valid, reliable, and appropriately normed, and have a wide scope of measurement, as previously described. In addition, as discussed later under Use of Validity Tests, current SSA policy precludes the purchase of validity tests except in rare cases, such as a court order.

Policy Relevant to Evaluations of Disability for Somatic Symptoms Disproportionate to Demonstrable Medical Morbidity

There are three distinct groups of applicants seeking disability compensation for somatic symptoms unaccompanied by demonstrable anatomical, biochemical, or physiological abnormalities: somatoform disorders (recently termed somatic symptom disorders in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders [DSM-5]); multisystem illnesses; and chronic idiopathic pain conditions.

In all three of these types of conditions—somatoform disorder, multisystem illness, and chronic pain—the credibility, reliability, validity, or accuracy of the reported symptoms and/or impairment may be called into question. This is due to the absence of objective evidence or biomarkers that could explain or substantiate the applicant’s report of subjective distress and disability. When relying on self-report of symptoms and impairment,

SSA policy states that applicants may not be found disabled solely on the basis of self-reported statements about pain or other symptoms (Social Security Act § 223(d)(5)(A), § 1614(a)(3)(D); 20 CFR 404.1508, 404.1529, 416.908, 416.929; SSA, 1996b, 2014g).

In cases where an individual’s self-reported symptoms, including pain, suggest a greater degree of impairment than expected based on the objective medical evidence alone, other corroborative information from treating and nontreating medical sources and other sources is considered. Such information may include information about the individual’s

SSA has issued guidance on its policy for evaluating claims involving chronic fatigue syndrome (CFS) (SSA, 2014g). This guidance explains how SSA determines the presence of a medically determinable impairment in an individual with CFS, including some of the possible medical signs and laboratory findings that would help to support such a finding. SSA then assesses whether the medically determinable impairment could reasonably be expected to produce the reported symptoms. In cases where objective medical evidence does not substantiate the person’s statements, SSA considers the same types of evidence described for pain and other symptoms. SSA will also make a finding about the credibility of the person’s statements as described in the following section.

Policy on the Evaluation of Credibility

Assessing Credibility of Statements About Pain and Other Symptoms

Given that symptoms—“individual’s own description[s] of his or her physical or mental impairment(s)”—are insufficient under SSA regulations “to establish the existence of a physical or mental impairment or that the individual is disabled,” the regulations provide a two-step process for evaluating statements about pain, fatigue, weakness, and other symptoms (SSA, 1996c). The first step is to determine whether the individual has a medically determinable impairment that could reasonably be expected to produce the symptoms. If so, the second step is to evaluate the intensity and persistence of the symptoms and their effect on the applicant’s ability to function and perform work-related activities.

Given the subjective nature of symptoms such as pain, fatigue, nervousness, and the like, “objective medical evidence”—such as medical signs and laboratory findings—does not always substantiate the severity of an impairment as experienced by individuals and expressed in their self-reported symptoms. If the objective medical evidence does not support an individual’s statements about the intensity, persistence, and limiting effects of the symptoms, the examiner must determine the credibility of the statements based on all of the information in the case record (SSA, 1996c).

When determining the credibility of an applicant’s statements about symptoms, SSA states the examiner must consider specific indicators of credibility such as:

• Consistency, both internally (i.e., with other statements by the applicant) and with other information in the record (e.g., objective medical evidence, third-party reports and observations);
• The extent to which objective medical evidence may inform conclusions about the intensity and persistence of reported symptoms, even if the latter are not objectively measurable; and
• The individual’s longitudinal medical record (history) of persistence and severity of reported symptoms.

SSA requires the examiner to articulate specific reasons for the credibility finding based on the medical and other evidence in the case record. It is important to note both that a credibility finding need not reflect complete acceptance or rejection of the individual’s statements (i.e., the statements may be found to be partially credible) and that credibility concerns alone do not rule out the presence of disability (SSA, 1996c).

Use of Validity Tests

With rare exceptions, such as a court order, current SSA policy precludes the purchase of (validity) tests¹⁹ to help inform determinations about the credibility of an individual’s statements or about possible malingering (SSA, 2012e, 2013). It is SSA’s position that “tests cannot prove whether a claimant is credible or malingering because there is no test that, when passed or failed, conclusively determines the presence of inaccurate self-reporting” (SSA, 2013, section D), although SSA acknowledges that the

_____________

¹⁹ Such tests include the following: Rey-15 Item Memory Test (Rey-II), Miller Forensic Assessment of Symptoms Test (M-FAST), Millon Clinical Multiaxial Inventory, Minnesota Multiphasic Personality Inventory (MMPI), Minnesota Multiphasic Personality Inventory-2 (MMPI-2), Malingering Probability Scale, Structured Interview of Reported Symptoms, Test of Memory Malingering, and Validity Indicator Profile (SSA, 2008, 2013).

results of such tests “can provide evidence suggestive of poor effort or of intentional symptom manipulation” (SSA, 2008). Nevertheless, SSA will consider, along with all other relevant evidence, the results of symptom validity tests (SVTs) that are already in the claimant’s file (SSA, 2013). According to a 2013 report from the Office of the Inspector General, SSA:

There appears to be some confusion or inconsistency among SSA’s statements regarding validity testing. On the one hand, SSA clearly rejects the purchase of performance validity tests (PVTs) and SVTs by DDS and consultative examiners with statements such as the following:

• “Malingering cannot be proven with tests”;
• “Malingering is one aspect of the larger sphere of inaccurate self-reporting”;
• “No test … conclusively determines the presence of inaccurate patient self-report”; and
• “Even a high likelihood of malingering does not preclude severe limitations resulting from a genuine medically determinable impairment.”²⁰

On the other hand, SSA acknowledges that validity test results can “provide evidence suggestive of poor effort or intentional symptom manipulation” and states that it will consider validity test results that are already in an applicant’s file, along with all other relevant evidence. In fact, the statement that no one test “conclusively determines the presence of inaccurate patient self-report” seems to run counter to SSA’s dedication to obtaining as much evidence as possible and taking account of all the information when making a disability determination. It is important to divorce the concept of “malingering” from that of validity testing. As introduced in the following section, and made clear later in this chapter and elsewhere in the report and appendixes, validity test results can speak to performance (on performance-based tasks) and to the consistency and accuracy of responses on self-report measures. However, they provide limited information about intentionality and none about motive. It is important, therefore, not to discount the potential usefulness of validity test results on the grounds

_____________

²⁰ Quotations are taken from SSA (2008).

that malingering cannot be proven with tests or that a high likelihood of malingering and the presence of severe limitations resulting from a genuine medically determinable impairment cannot coexist.

MALINGERING AND CREDIBILITY

Malingering Base Rates

As defined in Chapter 1, malingering is the intentional presentation of false or exaggerated symptoms, intentionally poor performance, or a combination of the two, motivated by external incentives (APA, 2015; Bush et al., 2005; Heilbronner et al., 2009). Base rates of “probable malingering and symptom exaggeration,”²¹ as reported in a 2002 survey of members of the American Board of Clinical Neuropsychology, vary depending on the alleged impairment (e.g., mild head injury, depressive or anxiety disorders, seizure disorders, vascular dementia), the context (e.g., personal injury or disability, criminal, medical or psychiatric), and the referral source (e.g., plaintiff, defense) (Mittenberg et al., 2002). All of these factors make direct comparisons of the reported rates difficult. For this reason, the discussion in this section focuses on studies of “malingering” in the disability context.

The studies described here suggest that anywhere from 19 to 68 percent of SSA disability applicants may be performing below their capability on cognitive tests or inaccurately reporting their symptoms. A number of factors may account for the vast range, including differences in what precisely is being reported, differences in the tests administered or the indicators (e.g., patterns of performance, inconsistencies among different sources of information) being used, and differences in the populations being examined. It is notable that a number of these articles refer to “malingering,” “probable malingering,” or “definite malingering” (see, e.g., Chafetz et al., 2007; Larrabee, 2007; Mittenberg et al., 2002; Samuel and Mittenberg, 2005). What is being reported, however, are either failure rates at different levels (e.g., below chance, at chance, below cut score, failure on two or more validity measures) on various PVTs or SVTs or other indicators, such as inconsistencies or discrepancies in the evidence.

_____________

²¹ Respondents were asked the extent to which each of the following supported such an assessment in their cases: “below empirical cut-off on forced-choice tests”; “below chance on forced-choice tests”; “below empirical cut-off on other malingering tests”; “pattern of cognitive test performance does not make neuropsychological sense (inconsistent with condition)”; “severity of cognitive impairment inconsistent with condition”; “implausible changes in test scores across repeated examinations”; “above validity scale cut-offs on objective personality tests”; “discrepancies among records, self-report, and observed behavior”; and “implausible self-reported symptoms in interview” (Mittenberg et al., 2002, p. 1102).

The following discussion, summarized in Table 2-2, focuses on the reported base rates of validity test failure in the context of disability claims and specifies what is being measured in each case.

In 1996, Griffin and colleagues reported on 167 SSA disability applicants alleging psychological impairment in Los Angeles County between December 1993 and December 1994 (Griffin et al., 1996). As part of their psychological evaluation, these applicants were administered the Composite Disability Malingering Index (CDMI), a research tool created from portions of the Minnesota Multiphasic Personality Inventory (MMPI), the M Test, the Millon Clinical Multiaxial Inventory-II, and the Beck Depression Inventory. Nineteen percent (n = 32) of the 167 applicants assessed scored at a level identified as “malingering.” The CDMI scores for this group more closely resembled those of a group of disability examiners who were instructed to malinger than those of the comparison group of psychologically disabled individuals with no incentive to malinger. The subgroup identified as “malingering” differed from the rest of the disability applicant group only in the presence of a self-reported history of substance abuse.

In their 2002 survey, Mittenberg and colleagues (2002) found a base rate of “probable malingering or symptom exaggeration,” as described in note 17, of approximately 30 percent (reported) to 33 percent (adjusted)²² for disability or worker’s compensation cases. The rate varied relative to the referral source, with patients referred by defense attorneys or insurers having a higher rate of “probable malingering or symptom exaggeration.” Their estimates were based on a total of 33,532 cases reported in surveys returned by 131 of 375 possible respondents among the 388 members of the American Board of Clinical Neuropsychology. Eleven percent of the cases involved disability or worker’s compensation (n = 3,688), 19 percent (n = 6,371) involved personal injury litigation, 4 percent (n = 1,341) involved criminal litigation, and 66 percent (n = 22,131) were medical or psychiatric cases not involving litigation or compensation. The reported base rate of “probable malingering or symptom exaggeration” in the last group was only 8 percent (Mittenberg et al., 2002, pp. 1095–1096).

In a sample of adult SSA disability applicants, Chafetz and Abrahams found that 13.8 percent scored below chance performance and 58.6 percent failed two or more validity indicators (Chafetz and Abrahams, 2005, reported in Larrabee, 2007). Miller and colleagues (2006) reported that more than 50 percent of 105 disability applicants failed “conservative criteria” for the Computerized Assessment of Response Bias.²³

_____________

²² The adjusted value is corrected to remove significant variation due to referral source.

²³ The information and data in this sentence have been revised from that provided in the prepublication version of the report.

TABLE 2-2 Summary of Reported Base Rates of Malingering

^a The information in this entry has been revised from that provided in the prepublication version of the report.

Chafetz and colleagues administered the Test of Memory Malingering (TOMM) or the Medical Symptom Validity Test (MSVT) to adult and child disability applicants, most with low cognitive functioning, who were referred for a psychological CE by the DDS (Chafetz et al., 2007). Based on their performance on the test, subjects’ performance was scored as “below chance,” “chance or below,” or “failing.” In this study, 55.8 percent of adults (n = 136) and 28.3 percent of children (n = 96) failed the TOMM, and 12.4 percent of adults and 8.7 percent of children scored below chance on the test. On the MSVT, 61.4 percent of adults (n = 58) and 37.0 percent of children (n = 27) failed, and 12.3 percent of adults and 7.4 percent of children scored below chance.

The same study was designed to validate a tool, the “DDS Malingering Rating Scale,” developed by the authors to help psychologists assess and inform DDSs about the validity of their findings (Chafetz et al., 2007).²⁴ The rating scale was validated against the TOMM and the MSVT and was found to correlate well with “formal tests and indicators of effort in adults and children” (Chafetz et al., 2007, p. 11). Fifty-one point six (51.6) to 58.9 percent of adults and 34.6 to 43.8 percent of children failed the DDS

_____________

²⁴ To the committee’s knowledge, the “DDS Malingering Rating Scale” has never been used or endorsed by any DDS agencies.

Malingering Rating Scale, and 20.5 to 30.4 percent of adults and 15.4 to 32.5 percent of children scored below chance (Chafetz et al., 2007, p. 10).

In a subsequent paper that draws on the research reported in Chafetz and colleagues (2007), Chafetz reports 67.8 percent of adults who were administered both the TOMM and the DDS Malingering Rating Scale failed at least one, 45.8 percent failed both, and 36.5 percent scored at or below chance. For adults who were administered both the MSVT and the rating scale, 68.4 percent failed at least one, 59.7 percent failed both, and 47.4 percent scored at or below chance on at least one of the SVT subtests. Sixty percent of children who were administered the TOMM and the rating scale failed at least one and 26.3 percent scored at or below chance. Of children who were administered the MSVT and the rating scale, 48 percent failed at least one, and 20 scored at or below chance on at least one of the SVT subtests (Chafetz, 2008).

In the context of SSA disability evaluations, it is important to note that even if an applicant performs below his or her capability on cognitive tests or inconsistently reports symptoms, neither scenario means the individual is not disabled. However, both scenarios suggest the need for additional assessment of the alleged impairment with the goal of making an accurate determination of disability. Doing so first requires identification of the individuals for whom additional assessment may improve the accuracy of the disability determination. As described in the section on assessing credibility, when a disability claim is based primarily on an applicant’s self-report of symptoms and statements about their intensity, persistence, and limiting effects, SSA relies on an assessment of the consistency of the self-report with all of the evidence in the claimant’s medical evidence record. As discussed, SSA policy currently precludes the purchase of validity tests by SSA (e.g., as part of a psychological CE). One question is whether the results of this type of standardized test could contribute to the evidence available for assessment. The following section discusses the potential value of adding standardized data collection and interpretation to clinical data collection and evaluation.

The Benefits of Mechanical Data Collection and Actuarial Data Interpretation

A robust literature demonstrates that people, including experts, are systematically overconfident in their ability to perform a wide range of tasks (Moore and Healy, 2008), from investing in the stock market (Scheinkman and Xiong, 2003) to estimating their level of general knowledge (Juslin, 1994; Oskamp, 1965). This overconfidence exists in large part because human judgment is influenced by biases that operate outside of conscious awareness (Kahneman, 2011). People believe they come to judgments by

rationally weighing evidence, unaware that other psychological forces are also influencing them.

This overconfidence extends to the judgment of practicing psychologists with obvious consequences for the accuracy of psychological evaluation (Oskamp, 1965). Clinicians may rely on clinical judgment alone to determine the degree of effort put forth on performance-based cognitive and behavioral tests and the credibility of an examinee’s self-report, even though research has shown that when people have been coached to exaggerate the symptoms of neurocognitive impairment, most clinicians failed to detect such malingering (Faust et al., 1988a,b; Heaton et al., 1978; Oldershaw and Bagby, 1997).

The literature comparing clinical versus actuarial (statistical) judgment suggests the best approach will (1) collect both clinical and structured data, and (2) combine these data using actuarial methods. Of course, considerable research is needed to establish the exact actuarial approach to be used.

Defining Terms

Data collection Medical professionals often evaluate patients using a combination of what Wedding and Faust call clinical and mechanical data (Wedding and Faust, 1989). Clinical data collection includes all testing and examining that is variable depending on how the clinician performs the exam and/or on which aspects of the exam the clinician chooses to perform. For example, clinicians may interview patients to elicit their description of the symptoms of their illness; alternatively, clinicians may perform a physical exam. By contrast, mechanical data collection involves the use of standardized testing where the data collection is structured and the method typically does not vary from patient to patient. For example, if clinicians order a serum sodium level or MMPI tests on their patients, they are collecting mechanical data.

It should be noted that mechanical data collection is not completely divorced from clinical expertise. For example, clinicians may need to determine which mechanical data are relevant to collect in a given patient, making a judgment about whose diagnosis will be aided by a serum sodium level or an MMPI. In addition, the administration of mechanical tests can be affected by clinical skill. For example, a clinician who draws a patient’s blood above an IV site will get a false sodium level. Similarly, a clinician who administers an MMPI test after the patient has been exhausted by previous examinations may also be collecting the data in a way that will reduce the value and accuracy of the test results.

Data interpretation Once data have been collected—whether clinical data, mechanical data, or some combination of both—they must be interpreted

to determine whether the patient has a specific health condition and to estimate how severe that condition is. Data interpretation generally takes one of two approaches: clinical or actuarial. In clinical data interpretation, a clinician looks at all the data and makes a judgment. (“Based on your age, family history, chest pain, and ECG [electrocardiogram], I think you are having a heart attack.”) In actuarial data interpretation, data are entered into a diagnostic program and weighed according to a statistical procedure. (“The presence of chest pain, given your age, family history, and ECG changes, yields a risk score of x, which estimates the probability of a heart attack to be y.”)

What Are the Evaluative Alternatives?

There is a range of possible approaches to the evaluation of people complaining of behavioral or cognitive impairments. At one extreme is a purely clinical evaluation, whereby expert clinicians collect clinical data from patients and then interpret what these data mean. In this example, no mechanical data are collected, and the judgment is not made actuarially. A more common approach is a clinical interpretation of mixed data, whereby a clinician examines clinical data on the patient (some combination of exam and interview) and also performs some standardized “mechanical” tests, perhaps administering an MMPI. Then the clinician interprets this combination of data to make a judgment about the person’s condition. Studies suggest that both of these approaches—the purely clinical one and the clinical interpretation of mixed data—are typically less reliable and valid than approaches using actuarial methods to interpret that data (Ægisdóttir et al., 2006). If several pieces of clinical and mechanical data are available, for example, actuarial combination of this data performs better than clinical interpretation (Dawes et al., 1989). In fact, actuarial combination of just clinical data typically performs better than clinical interpretation of all the data. In short, actuarial combination of clinical data, mechanical data, and especially of both clinical and mechanical data performs better than clinical interpretation of clinical data, mechanical data, or even both kinds of data.

Why Are Actuarial Methods Controversial?

It is difficult for many clinicians to believe that an inflexible rule (“3 points for chest pain, 2 points for family history and heart disease, 2 points for change in the ST segment of the ECG leads to…”) would perform better than an experienced clinician who could take advantage of information not included in the actuarial formula. Indeed, some clinicians recoil at actuarial methods for being too impersonal; for treating patients like numbers and not like unique individuals. Others criticize actuarial methods for ignoring

useful information available to clinicians. A famous criticism of actuarial methods is known as the “broken leg problem.” In one version of this, Professor A goes to the movies almost every Tuesday night. Knowing that today is Tuesday, an actuarial table might predict that the probability of Professor A going to the movie tonight is 0.9. However, you might know that Professor A just broke his leg and cannot get out of the house. You will have a much more accurate estimate of tonight’s chance of him going to the movie than the actuarial approach (Salzinger, 2005).

The psychological power of this counterexample is that it makes it seem obvious that a clinician, given actuarial information, can always improve on actuarial judgment by using additional information not available to the actuarial formula. In practice, however, few cases are as clear-cut as the broken leg example. Most additional information will not dramatically change likelihood estimates derived from validated actuarial methods. In addition, even when additional relevant data are available, clinicians may not make proper use of the data. They may give the data too much or too little weight (Dawes, 1979).

In summary, clinicians are trained to collect clinical data from patients and to make decisions about which mechanical data will aid in diagnoses as well as to interpret these clinical and mechanical data. However, clinicians are generally not as good at interpreting those data as are established actuarial methods (Grove and Meehl, 1996; Grove et al., 2000; Meehl, 1954). There is evidence that the use of clinical judgment alone to assess whether an individual is exerting sufficient effort on performance-based tests or is providing an accurate self-report of symptoms is unreliable (Faust et al., 1988a,b; Heaton et al., 1978; Oldershaw and Bagby, 1997), making it important for the evaluator to collect and consider relevant mechanical data along with other objective data in making such assessments.

USE OF PSYCHOLOGICAL TESTS IN NON-SSA DISABILITY EVALUATIONS

To better understand the potential role of standardized psychological testing, including validity testing, for SSA disability determinations, the committee looked at current practices surrounding the use of psychological testing in several other settings that involve, or might involve, an element of secondary gain. The VA provides disability benefits to veterans who qualify based on injuries or disease incurred or aggravated during active military service or postservice disabilities that are related or secondary to disabilities occurring during service or are presumed to be related to circumstances of military service. The U.S. military assesses active duty military personnel for fitness for return to duty following injury. Private disability insurance programs determine whether claimants under their plans meet the criteria

to receive benefits. The automobile insurance industry determines claims of injury following auto accidents. Finally, the forensic setting (i.e., criminal and civil judicial contexts) includes litigation for personal injury and determinations of competency to stand trial. Common to all of these settings is assessment of an individual’s alleged impairments to determine whether the individual qualifies for an outcome that may benefit him or her (e.g., disability benefit, restriction of military duty, compensation for injury, incompetence to stand trial). Despite this common element, the context of the settings—the purposes for which the assessments are being conducted—differ in important ways as discussed in the following sections.

Military and Veterans Affairs

Mental and behavioral health conditions have become more prevalent and consume a larger portion of the military and VA budget than they did 5 years ago. Within the past 10 years, the VA has reached consensus about the compensability of behavioral health conditions (e.g., posttraumatic stress disorder [PTSD]).

Significant progress has been made in defining mental and behavioral diagnoses. Both the military and the VA have measures of mental and behavioral health, and both evaluation systems address function as a key determinant for disability, although for somewhat different purposes, as described in the following sections.

Military²⁵

There are significant differences between policies and procedures followed by SSA and the military. In contrast to disability evaluations for SSA and the Veterans Benefits Administration (VBA), discussed in the following section, military assessments for mental and behavioral health are performed to assess combat or duty readiness. Assessing whether an individual is capable of performing his or her duty may be an issue of safety not only for the individual but also for others.

Fitness for duty and return-to-duty determinations are made by medical evaluation boards and physical evaluation boards. Mental health providers serve as consultants to the boards, providing them with reports of diagnostic impressions, assessment of degree of impairment and impact on military duty performance, prognosis, and recommendations. In contrast to SSA and the VBA, evaluations in the military are often performed by therapists and care professionals who are not “interrogators” but are considered

_____________

²⁵ Much of the information in this section is drawn from the presentation to the committee by Robert Seegmiller (2014).

advocates and treating professionals, which may present a conflict with respect to treatment goals versus determinations of fitness for duty. It also should be noted that Army behavioral health professionals “diagnose and treat and should not be in an adversarial role with patients in terms of disability processes” and “must approach with a soldier-centered focus that provides soldiers the benefit of the doubt.” Providers “on the whole do support the patient/soldier on face value and advocate in every way for them; however, [they] lose credibility with both medical personnel and line units when [they] fail to properly investigate and obtain collateral information” (U.S. Army Medical Command [MEDCOM] Behavioral Health Training Day, June 12, 2012, reported in Seegmiller, 2014).

Evaluations typically include review of medical records, consideration of premorbid functioning (the Armed Services Vocational Aptitude Battery), clinical interview and behavioral observations, and information from collateral sources. Psychological or neuropsychological testing is required in cases involving reported traumatic brain injury (TBI), but not always in cases involving PTSD. The selection of specific tests is left to the discretion of the clinician performing the evaluation, as is the use of PVTs and SVTs, although most providers, particularly psychologists and neuropsychologists, recognize the importance of their use.

A previous Office of the Surgeon General (OTSG)/MEDCOM policy memo on the optimal use of psychological and neuropsychological assessment, notes (1) “psychological and neuropsychological assessments are valuable tools in quantifying patient deficits, clarifying diagnoses, informing treatment, and in making decisions regarding a soldier’s continued fitness for military service” and (2) “certain clinical tests in use by neuropsychology are designed to evaluate level of effort on the part of the test-taker. Poor effort on cognitive symptom validity measures means only that the data is not valid to be fully interpreted, and invalid data can be due to a range of causes other than malingering” (Policy Memo 11-076: Optimal Use of Psychological/Neuropsychological Assessment [21 Sept 2011–2013], reported in Seegmiller, 2014). “Poor effort on psychological/neuropsychological tests does not equate malingering, which requires proof of intent, per OTSG/MEDCOM Policy 11-076. In addition, this diagnosis requires the signatures of two credentialed care providers, including a supervisor, Department Chief, or Deputy Commander for Clinical Services” (OTSG/MEDCOM Policy Memo 12-035: Policy Guidance on the Assessment and Treatment of Post-Traumatic Stress Disorder [10 Apr 12 thru 10 Apr 14], reported in Seegmiller, 2014).

In his discussion with the committee, Dr. Robert Seegmiller (2014) asserted that SVTs and PVTs are critical tools that provide valuable information about the validity of an individual’s test results. When making decisions and recommendations about whether soldiers are fit for duty or

whether they need disability, Seegmiller noted the importance of ensuring that one has good information in order to make the decision and recommendation that is the fairest for them and best for the system in terms of returning to work or not. However, such tests are only one type of tool: clinician’s performing the evaluation also review the individual’s medical records, conduct a clinical interview, make behavioral observations, gather collateral information, and the like, and consider the consistency of all of the information with what the patient is reporting.

Veterans Health Administration²⁶

The VBA is responsible for administering and delivering an array of federally authorized benefits and services to eligible veterans and their dependents and survivors. In fiscal year 2012, 3,536,802 veterans received compensation benefits. PTSD was the third most prevalent service-connected disability among veterans receiving compensation at the end of fiscal year 2012, and TBI has been widely reported as the hallmark injury of the wars in Afghanistan and Iraq. To be eligible for disability compensation, a veteran must have served under conditions other than dishonorable, and the disability must not be the result of misconduct by the veteran. In contrast to the military setting, in which service members are assessed in terms of fitness for duty, veterans’ assessments are performed with the recognition that there is responsibility to care for individuals who served in the military.

Disability compensation is paid monthly and varies according to the degree of disability and the number of dependents. The rate of compensation is graduated from 10 percent to 100 percent disabling, in increments of 10 percent, according to the combined degree of the veteran’s disabilities. This differs from SSA, which determines an individual to be either disabled (100 percent) or not. Also unlike SSA, recipients of veterans’ disability benefits may work with no limit on their earnings.

Disability examinations are conducted by full-time employees of the Veterans Health Administration (VHA), fee-basis staff, and contracted staff. Initial evaluations can be conducted by

_____________

²⁶ Much of the information in this section is drawn from the presentation to the committee by Stacey Pollack (2014).

Under the close supervision of a board-certified or board-eligible psychiatrist or licensed doctoral-level psychologist, reviews and increase evaluations can be conducted by licensed clinical social workers, nurse practitioners or clinical nurse specialists, and physician assistants (VHA Directive 2012-021, August 27, 2012).

The VHA requires all examiners to complete general online training regarding compensation and pension (C&P). Some specialty examiners are required to take additional training related to specific disabilities (e.g., PTSD).

Examiners conducting C&P examinations for mental disorders are instructed to:

• Diagnose mental disorders, including personality disorders, using the nomenclature in the most current edition of the Diagnostic and Statistical Manual of Mental Disorders; …
• Determine when clinician-administered psychometric testing is necessary and integrate the results of such testing into the examination reports; …
• When necessary, comment on the significance of the veteran’s prior mental health assessments (as reported) with respect to symptoms, occupational history, social history, and global assessment of functioning. (VHA Directive 2012-021, August 27, 2012)

For all initial PTSD disability evaluations, the examiner is instructed to review the veteran’s claims file (C-file) or any other available medical records prior to conducting the examination. For an Integrated Disability Examination System (IDES) examination, the examiner is required to review the service member’s medical records. Examiners are instructed to obtain results from all pertinent studies, evaluations, and tests, and order or perform any further studies, evaluations, or tests needed to diagnose a

mental disorder before completing their report. In addition, examiners must assess the individual for functional impairment. The examination report is used along with all other evidence to determine what level of compensation may be awarded to the veteran or service member.

VHA policy requires mental health examiners to review all records provided by VBA as part of a comprehensive evaluation. These records typically include the claimant’s medical record. If there are psychological tests in the claimant’s medical record, these should be reviewed as part of the evidence used in a comprehensive examination. The option to order additional psychological tests, including validity tests, is left to the discretion of the examiner. VA policy neither requires nor prohibits the ordering or use of any specific tests or categories of tests to evaluate any mental health condition.

Private Disability Insurance

Unum is the largest commercial disability insurer in the United States for both short-term and long-term disability. The committee looked to its processes to gain an understanding of how private disability insurers approach the use of psychological testing in adjudicating claims.²⁷ In evaluating a claim, examiners, who are clinicians, are required to consider all of the information in the claimant’s file, including the results of previously administered psychological and neuropsychological tests. Examiners will attempt to acquire the raw test materials—the actual reports, the actual scores, the actual tests with the questions and answers—to analyze those data independently and determine whether they match the conclusions of the clinician who administered the tests. The examiners also are mandated to speak to the claimant’s attending physicians.

If an independent medical examination (IME), an umbrella term that includes psychological, neuropsychological, or psychiatric examinations, is needed to provide additional information, the practitioner conducting the examination may administer any psychological tests or measures he or she feels are most valid based on current scientific literature and research. IME examiners are required to include peer-reviewed, scientifically validated measures of symptom and performance validity in their evaluations.²⁸ Validity of test results is addressed through a three-tiered system, formal

_____________

²⁷ The information in this section is drawn from the presentation to the committee by Thomas McLaren (2014).

²⁸ This is consistent with the findings of the SSA Office of the Inspector General, which reports on the practices of three private disability insurance providers, all of which allow the purchase and use the results of validity tests in their disability claims processes. All three companies also indicated that validity test results are just one piece of data they consider when evaluating claims (Office of the Inspector General, 2013). The names of the companies are not released in the report.

effort by stand-alone validity measures, consideration of imbedded validity measures, and an examination of the pattern of testing—meaning, whether it makes neurologic or medical sense for the condition being evaluated.

Although validity testing is required by Unum, the results of such testing are data points, which when taken in isolation can be misconstrued. For this reason, examiners are mandated to look at all of the information collectively. Invalid results on validity measures indicate that the remaining test results are not valid for clinical interpretation. In such cases, the IME or claims examiner would seek information from other sources.

After collecting and examining all the data relevant to the claim, the claims examiner balances the data to make a decision on the outcome and the claimant’s restrictions and limitations—i.e., what the person is unable to do and what the person should not do.

Forensic Assessment: Criminal and Civil Judicial Contexts

At its most basic, the role of the legal system is to adjudicate disputes based on factual evidence. To achieve this goal, the courts rely on the collection of facts from a multitude of sources that are directly relevant to a specific legal question. One such source of information is the testimony of witnesses, who may provide the court with factual evidence based on personal knowledge of the matter but are prohibited from testifying based on their own opinions or analysis (Federal Rules of Evidence, Rule 602). However, under certain circumstances, the law does allow for the provision of opinions by an expert based on facts or data in the case (Federal Rules of Evidence, Rule 703). According to Rule 702 of the Federal Rules of Evidence:

With the requirement that the expert witness be able to provide information that is directly relevant to the question at hand, such witnesses can come from a variety of fields, including mental health. Once established as an expert witness, a mental health professional (i.e., psychologist, psychiatrist, or social worker) may provide expert opinion to assist in answering the legal question at hand.

Psychological assessments may be used in a variety of contexts and at all stages of the judicial process. For example, one of the primary uses for psychological assessments is to assess competency. During pretrial information gathering, this includes competencies such as whether a defendant was competent to consent to search and seizure or to confess, or to answer questions regarding mental state at the time of the offense. Similarly, psychological assessments may be used during the trial phase to answer questions related to competence to plead guilty, waive the right to counsel, testify, or refuse an insanity defense. Following a guilty verdict, psychological assessment may help answer questions related to competency to be sentenced or executed. In civil contexts, psychological assessments may be used to help answer questions related to civil commitment, compensation for mental injuries, or questions of competency, such as for guardianship, making treatment decisions, or consenting to research.

Psychological assessment for the courts is typically based on a variety of information sources and methods of data collection, including psychometric testing. Establishing symptom, performance, and response validity²⁹ is of particular importance in forensic contexts, as the potential for secondary gain may lead to examinee attempts to minimize, exaggerate, or feign problems (Bush et al., 2014). As noted in a statement from the Association for Scientific Advancement in Psychological Injury and Law, “Measures of performance and symptom validity are still in their relative infancy … [and] methodological difficulties exist in validity assessment research” (Bush et al., 2014; see also Chapters 4 and 5 of this report). For example, Bush and colleagues (2014) note there are few PVT manuals or articles that provide data on test-retest reliability on how reliably volunteers fake poor performance or simulate performance of actual examinees in simulation studies used to create cut-off scores. In addition, some comparison groups consist of mixed patient samples or populations that are dissimilar to an examinee and may not allow for appropriate comparisons. Finally, such tests do not necessarily speak to the intentionality behind invalid results, which may be generated consciously or unconsciously. Even in cases in which there is evidence of intentionally poor performance, the test results alone do not explain why the examinee did so (Bush et al., 2014).

Although the results of psychometric testing may play a crucial role in the formulation of a mental health professional’s expert opinion for the courts, it is important to note that such tests are rarely used in isolation,

_____________

²⁹ The Association for Scientific Advancement in Psychological Injury and Law has identified a third type of validity important to forensic psychological assessment, termed response validity, as “the accuracy of the examinee’s responses to autobiographical questions (e.g., educational history, vocational history, legal history) and questions pertaining to the legal matter in question (e.g., the nature of, and events surrounding, an injury, crime, or traumatic event)” (Bush et al., 2014, p. 199).

with most tests requiring some degree of subjective interpretation (Cohen and Malcolm, 2005). As with psychometric testing, evaluation of validity also should not rely on test scores alone, but rather, employ a multimethod approach (Bush et al., 2014). In addition to psychometric testing, forensic psychological assessment is typically based on a variety of other information sources, such as clinical interview, observational methods, and interviews with third parties.

International Community

Canada

The Canada Pension Plan (CPP) provides disability benefits to eligible individuals using much the same criteria in its disability determination process as SSA does (Government of Canada, 2014) As in the United States, there are a number of different settings in which disability determinations are made. Settings in addition to the CPP include the Worker Safety Insurance Board, Veterans Affairs Canada, and the auto insurance industry. Psychologists and neuropsychologists do not work under the Canadian national health care system. As a result, they work in a number of other settings, such as auto insurance.

Brian Levitt (2014) presented to the committee on the use of psychological testing under private auto insurance in the province of Ontario as well as tort law in Ontario. In this setting as well, the decision of whether to administer psychological tests and, if so, which particular test to use is determined by the individual psychologists according to the practice standards in that area of inquiry. The Canadian Academy of Psychologists and Disability Assessment standards related to psychological testing include the following:

• A psychologist shall employ standardized psychometric tests whenever possible;
• Psychologists whenever possible shall employ psychometric procedures that measure response bias and symptom validity; and
• Psychologists shall address any apparent discrepancies between the results of psychometric tests and other information.

These standards are consistent with the message that the use of validity tests is important, but they constitute only one piece of data, which must be interpreted in the context of all the other information.

Europe

Merten and colleagues (2013) have reported that large-scale research on and use of SVTs and PVTs in Europe followed that in the United States by about a decade, beginning in earnest in the early 2000s. As in the United States, the setting or context (forensic, clinical, etc.) seems to matter (Dandachi-FitzGerald et al., 2013; McCarter et al., 2009; Merten et al., 2013). It is important to note that in the study by Dandachi-FitzGerald and colleagues (2013) the definition of SVT was left to the respondent. Everything from discrepancies between records and observed behavior, to more “objective” scales on personality and effort tests was included, making it very difficult to interpret the findings regarding the percentage of medical professionals using SVTs when contracted to assess work capacity due to claims of psychological disability. There also appear to be differences in SVT and PVT use across European countries, with practitioners in the Netherlands and Norway reporting the greatest use of such tests (Merten et al., 2013).

Closing Comments

SSA, the U.S. military, the VBA, private disability insurance providers, and forensic assessment in civil and criminal judicial contexts have different goals, needs, and approaches to the evaluation and determination of disability (see Table 2-3). All share common elements, including identification of the presence of impairment and evaluation of its effect on the individual’s ability to function.

Although the use of psychological testing must be understood in the context of each system’s goals, each of the systems encourages a comprehensive evaluation, as determined by the evaluator, in an effort to answer these questions and each permits a broad range of evaluations. Whether to order psychological tests and the selection of which tests to administer are left to the discretion of the professional performing the evaluation or examination. With the exception of SSA, all of the systems permit, or in some cases require, the use of validity testing to provide information about the validity of the results of other psychological tests being administered. Nevertheless, all agree that although validity tests yield important information, the results of such tests are only one piece of data that needs to be assessed and interpreted in the context of all the other information available.

TABLE 2-3 Psychological Testing in Different Settings

NOTE: DDS = Disability Determination Services; IDES = Independent Disability Examination System; NP = nurse practitioner; PA = physician assistant; PTSD = posttraumatic stress disorder; PVT = performance validity test; SVT = symptom validity test; TBI = traumatic brain injury.

^a Some require standard tests, such as the AMA Guide (see, for example, Rondinelli, 2008).

FINDINGS

• There currently is great variability in allowance rates for both SSI and SSDI among states that is not fully accounted for by differences in the populations of applicants. There also is great variability in the disability determination appeal rulings among ALJs within and across states.
• Each state DDS agency, within the confines of SSA policy, issues its own rules regarding the tests that may be purchased as part of a CE. For this reason, there is variation among states about when and which standardized psychological tests can be purchased, with the exception of PVTs and SVTs, which are precluded from purchase by SSA.
• There currently are no data on the rates of false positives and false negatives in SSA disability determinations.
• Identification and documentation of the presence and severity of medically determinable mental impairments at Step 2 of SSA’s disability determination process could be informed by results of standardized psychological tests.
• Identification and assessment of the severity of work-related functional impairment relevant to disability evaluations at the listing level (Step 3) and to mental residual functional capacity (Steps 4 and 5) are other points in SSA’s disability determination process that could be informed by results of standardized psychological tests.
• Consultative examinations may be ordered by DDS examiners or ALJs to supplement evidence in a claimant’s case record. Psychological tests could be administered as part of a CE.
• In some cases, SSA disability examiners must evaluate the credibility of statements by individuals about the intensity and persistence of their symptoms and the effect on the individual’s ability to function and perform work-related activities.
• Current data on the prevalence of inconsistent reporting of symptoms or performing below one’s capability on cognitive tests among SSDI and SSI applicant populations are limited.
• Current SSA policy precludes the purchase of (validity) tests—e.g., MMPI-2 and TOMM—to help inform determinations about the credibility of an individual’s statements or about possible malingering.
• There is inconsistency among SSA’s statements on validity testing:
  • Results can “provide evidence suggestive of poor effort or intentional symptom manipulation.”

• • “Malingering cannot be proven with tests”; “malingering is one aspect of the larger sphere of inaccurate self-reporting.”
  • “No test … conclusively determines the presence of inaccurate patient self-report.”
  • “Even a high likelihood of malingering does not preclude severe limitations resulting from a genuine medically determinable impairment.”
• Clinicians generally are not as good at interpreting clinical and mechanical data as are established actuarial methods.
• Each of the systems reviewed leave the question of whether to order psychological tests and the selection of which tests to administer to the discretion of the professional performing the evaluation or examination. With the exception of SSA, all of the systems permit, or in some cases require, the use of validity testing to provide information about the validity of the results of other psychological tests being administered. Nevertheless, all agree that although validity tests yield important information, the results of such tests are only one piece of data that needs to be assessed and interpreted in the context of all the other information available.

REFERENCES

Ægisdóttir, S., M. J. White, P. M. Spengler, A. S. Maugherman, L. A. Anderson, R. S. Cook, C. N. Nichols, G. K. Lampropoulos, B. S. Walker, G. Cohen, and J. D. Rush. 2006. The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist 4(3):341-382.

APA (American Psychological Association). 2015. Guidelines and principles for accreditation of programs in professional psychology: Quick reference guide to doctoral programs. http://www.apa.org/ed/accreditation/about/policies/doctoral.aspx (accessed January 20, 2015).

Autor, D. H., and M. G. Duggan. 2003. The rise in the disability rolls and the decline in unemployment. Quarterly Journal of Economics 118(1):157-205.

Black, D., K. Daniel, and S. Sanders. 2002. The impact of economic conditions on participation in disability programs: Evidence from the coal boom and bust. American Economic Review 92(1):27-50.

Burkhauser, R., J. S. Butler, and R. Weathers II. 2002. How policy variables influence the timing of applications for Social Security Disability Insurance. Social Security Bulletin 64(1):52-83.

Bush, S. S., R. M. Ruff, A. I. Tröster, J. T. Barth, S. P. Koffler, N. H. Pliskin, C. R. Reynolds, and C. H. Silver. 2005. Symptom validity assessment: Practice issues and medical necessity. NAN Policy & Planning Committee. Archives of Clinical Neuropsychology 20(4):419-426.

Bush, S. S., R. L. Heilbronner, and R. M. Ruff. 2014. Psychological assessment of symptom and performance validity, response bias, and malingering: Official position of the Association for Scientific Advancement in Psychological Injury and Law. Psychological Injury and Law 7(3):197-205.

Chafetz, M. D. 2008. Malingering on the Social Security disability consultative exam: Predictors and base rates. The Clinical Neuropsychologist 22(3):529-546.

Chafetz, M., and J. Abrahams. 2005. Green’s MACT helps identify internal predictors of effort in the social security disability exam. [Abstract.] Archives of Clinical Neuropsychology 20(7):889-890.

Chafetz, M. D., J. P. Abrahams, and J. Kohlmaier. 2007. Malingering on the Social Security disability consultative exam: A new rating scale. Archives of Clinical Neuropsychology 22(1):1-14.

Cohen, A., and C. Malcolm. 2005. Psychological assessment for the courts. In Psychology and law, edited by C. Tredoux, D. Foster, A. Allan, A. Cohen, and D. Wassenaar. Lansdowne, South Africa: Juta and Co. Ltd.

Dandachi-FitzGerald, B., R. W. Ponds, and T. Merten. 2013. Symptom validity and neuropsychological assessment: A survey of practices and beliefs of neuropsychologists in six European countries. Archives of Clinicial Neuropsychology 28(8):771-783.

Dawes, R. 1979. The robust beauty of improper linear models in decision making. American Psychologist 34(7):571-582.

Dawes, R., D. Faust, and P. Meehl. 1989. Clinical versus actuarial judgments. Psychological Science 243(4899):1688-1674.

Duggan, M., and S. Imberman. 2008. Why are disability rolls skyrocketing? The contribution of population characteristics, economic conditions, and program generosity. In Health at older ages: The causes and consequeses of declining disability among the elderly, edited by D. Cutler and D. Wise. Chicago, IL: University of Chicago Press.

Faust, D., K. Hart, and T. Guilmette. 1988a. Pediatric malingering: The capacity of children to fake believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology 56(4):578-582.

Faust, D., K. Hart, T. Guilmette, and H. Arkes. 1988b. Neuropsychologists’ capacity to detect adolescent malingerers. Professional Psychology: Research and Practice 19(5):508-515.

Government of Canada. 2014. How applications for disability benefits are assessed. http://www.servicecanada.gc.ca/eng/services/pensions/cpp/disability/benefit/assessment.shtml (accessed January 4, 2015).

Griffin, G. A., J. Normington, R. May, and D. Glassmire. 1996. Assessing dissimulation among Social Security disability income claimants. Journal of Consulting and Clinical Psychology 64(6):1425-1430.

Grove, W. M., and P. E. Meehl. 1996. Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law 2(2):293-323.

Grove, W. M., D. H. Zald, B. S. Lebow, B. E. Snitz, and C. Nelson. 2000. Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment 12(1):19-30.

Heaton, R. K., H. H. Smith, R. A. Lehman, and A. T. Vogt. 1978. Prospects for faking believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology 46(5):892-900.

Heilbronner, R. L., J. J. Sweet, J. E. Morgan, G. J. Larrabee, S. R. Millis, and Conference Participants. 2009. American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist 23(7):1093-1129.

Heiser, N. 2014. Disability Determination Services panel discussion with the committee. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, August 11, 2014, Washington, DC.

Juslin, P. 1994. The overconfidence phenomenon as a consequence of informal experimenter-guided selection of almanac items. Organizational Behavior and Human Decision Processes 57(2):226-246.

Kahneman, D. 2011. Thinking fast and slow. New York: Farrar, Straus, and Giroux.

Kreider, B. 1999. Latent work disability and reporting bias. Journal of Human Resources 34(4):734-769.

Larrabee, G. J. 2007. Introduction: Malingering, research designs, and base rates. In Assessment of malingered neuropsychological deficits, edited by G. J. Larrabee. New York: Oxford University Press.

Laurence, B. 2015. Third level of appeal for disability: Appeals council & remands. http://www.disabilitysecrets.com/appeals-council.html (accessed October 20, 2014).

Levitt, B. 2014. Psychological disability evaluations under the Ontario auto insurance system and Ontario tort law. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, June 25, 2014, Washington, DC.

McCarter, R. J., N. H. Walton, D. N. Brooks, and G. E. Powell. 2009. Effort testing in contemporary U.K. neuropsychological practice. The Clinical Neuropsychologist 23(6):1050-1066.

McLaren, T. 2014. Use of performance and symptom validity assessment within the independent disability insurer context. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, June 25, 2014, Washington, DC.

Meehl, P. E. 1954. Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis, MN: University of Minnesota Press.

Merten, T., B. Dandachi-FitzGerald, V. Hall, B. A. Schmand, P. Santamaría, and H. González-Ordi. 2013. Symptom validity assessment in European countries: Development and state of the art. Clínica y Salud 24(3):129-138.

Miller, L. S., M. C. Boyd, and A. Cohn. 2006. Prevalence of sub-optimal effort in disability applicants. [Abstract.] Journal of the International Neuropsychological Society 12(S1):159.

Mittenberg, W., C. Patton, E. M. Canyock, and D. C. Condit. 2002. Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology 24(8):1094-1102.

Moore, D., and P. Healy. 2008. The trouble with overconfidence. American Psychological Association 115(2):502-517.

Morton, D. 2014. Social Security disability: Four levels of appeal. http://www.nolo.com/legal-encyclopedia/social-security-disability-appeal-levels-32398.html (accessed March 27, 2015).

Office of the Inspector General, SSA (Social Security Administration). 2013. The Social Security Administration’s policy on symptom validity tests in determining disability claims. Washington, DC: SSA. http://oig.ssa.gov/sites/default/files/audit/full/pdf/A-08-13-23094. pdf (accessed March 27, 2015).

Oldershaw, L., and M. Bagby. 1997. Children and deception. New York: Guildford.

Oskamp, S. 1965. Overconfidence in case-study judgments. Journal of Consulting Psychology 29(3):261-265.

Pollack, S. 2014. VA policies and/or practices surrounding the use of psychological tests and symptom validity tests in the disability determination process. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, June 25, 2014, Washington, DC.

Price, J. H. 2014. Disability Determination Services panel discussion with the committee. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, August 11, 2014, Washington, DC.

Rondinelli, R. D., ed. 2008. AMA guides to the evaluation of permanent impairment, sixth edition. Chicago, IL: American Medical Association.

Rupp. 2012. Factors affecting initial disability allowance rates for the Disability Insurance and Supplemental Security Income programs: The role of the demographic and diagnostic composition of applicants and local labor market conditions. Social Security Bulletin 72(4):11-35. http://ssrn.com/abstract=2172488 (accessed February 4, 2015).

Rupp, K., and D. Stapleton. 1995. Determinants of the growth in the Social Security Administration’s disability programs—An overview. Social Security Bulletin 58(4):43-70.

Salzinger, K. 2005. Clinical, statistical, and broken-leg predictions. Behavior and Philosophy 33:91-99.

Samuel, R. Z., and W. Mittenberg. 2005. Determination of malingering in disability evaluations. Primary Psychiatry 12(12):60-68.

Scheinkman, J. A., and W. Xiong. 2003. Overconfidence and speculative bubbles. Journal of Political Economy 111(6):1183-1220.

Seegmiller, R. 2014. Use of psychological tests, including PVTs and SVTs, in select populations: The U.S. military. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, June 25 2014, Washington, DC.

Soss, J., and L. R. Keiser. 2006. The political roots of disability claims: How state environments and policies shape citizen demands. Political Research Quarterly 59(1):133-148.

SSA (Social Security Administration). 1996a. SSR 96-3p: Policy interpretation ruling. Titles II and XVI: Considering allegations of pain and other symptoms in determining whether a medically determinable impairment is severe. http://www.socialsecurity.gov/OP_Home/rulings/di/01/SSR96-03-di-01.html (accessed August 20, 2014).

SSA. 1996b. SSR 96-4p: Policy interpretation ruling. Titles II and XVI: Symptoms, medically determinable physical and mental impairments, and exertional and nonexertional limitations. http://www.socialsecurity.gov/OP_Home/rulings/di/01/SSR96-04-di-01.html (accessed October 3, 2014).

SSA. 1996c. SSR 96-7p: Policy interpretation ruling Titles II and XVI: Evaluation of symptoms in disability claims: Assessing the credibility of an individual’s statements. http://www.socialsecurity.gov/OP_Home/rulings/di/01/SSR96-07-di-01.html (accessed October 3, 2014).

SSA. 2008. National Q&A, 08-003 rev 2, do tests of malingering have any value for SSA evaluations? Washington, DC: SSA.

SSA. 2009. DI 22511.005 Documenting the impact of a medically determinable mental impairment on an individual’s ability to work. Program Operations Manual System (POMS). https://secure.ssa.gov/apps10/poms.nsf/lnx/0422511005 (accessed January 30, 2015).

SSA. 2010. Revised medical criteria for evaluating mental disorders. Federal Register 75(160):51336-51368.

SSA. 2012a. DI 00115.001 Social Security Administration’s (SSA’s) disability programs. Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0400115001 (accessed August 20, 2014).

SSA. 2012b. DI 22501.001 Disability case development for medical and other evidence. Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0422501001 (accessed October 3, 2014).

SSA. 2012c. DI 22510.048 Pediatric consultative examination (CE) report content guidelines—Mental disorders. Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0422510048 (accessed October 3, 2014).

SSA. 2012d. DI 22511.007 Sources of evidence. Program Operations Manual System (POMS). https://secure.ssa.gov/apps10/poms.nsf/lnx/0422511007 (accessed December 30, 2014).

SSA. 2012e. Disability Determination Services adminsistrative letter no. 866: Consulative examinations malingering & credibility tests—Information. Washington, DC: SSA.

SSA. 2012f. Social Security testimony before Congress. Statement of Michael I. Astrue, Commisioner, Social Security Administration before the Committee on Ways and Means Subcommittee on Social Security, June 27, 2012. http://www.ssa.gov/legislation/testimony_062712.html (accessed October 20, 2014).

SSA. 2013. DI 22510.006 When not to purchase a consultative examination (CE). Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0422510006 (accessed October 3, 2014).

SSA. 2014a. Annual report of the Supplemental Security Income program. Baltimore, MD: SSA. SSA. 2014b. Annual statistical report on the Social Security Disability Insurance program, 2013. Washington, DC. SSA. http://www.ssa.gov/policy/docs/statcomps/di_asr (accessed February 24, 2015).

SSA. 2014c. DDS performance management report. Disability claims data. Consultative examination rates, fiscal year 2013. Data prepared by ORDP, ODP, and ODPMI. Data submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 8, 2014.

SSA. 2014d. Disability claims data (initial, reconsideration, continuing disability review) by adjudictive level and body system. SSDI, SSI, Concurrent, and Total Claims. Data prepared by ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 8, 2014.

SSA. 2014e. DI 22510.021 Consultative examination (CE) report content guidelines: Mental disorders. Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0422510021 (accessed October 3, 2014).

SSA. 2014f. DI 24515.008 Titles II and XVI: Considering opinions and other evidence from sources who are not “acceptable medical sources” in disability claims; considering decisions on disability by other governmental and nongovernmental agencies (SSR 06-03p). Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/0424515008 (accessed February 24, 2015).

SSA. 2014g. DI 24515.075 Evaluating claims involving Chronic Fatigue Syndrome (CFS). Program Operations Manual System (POMS). https://secure.ssa.gov/poms.nsf/lnx/ 0424515075 (accessed December 16, 2014).

SSA. 2014h. National data: Title II—SSDI, Title XVI—SSI, & concurrent Title II/XVI initial disability determinations. By regulation basis code for adults and children (reason for decision), fiscal year 2013. Data submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 23, 2014.

SSA. 2014i. Open government initiative. Data on combined Title II disability and Title XVI blind/disabled average processing time (in days) (excludes technical denials). http://www.ssa.gov/open/data/Combined-Disability-Processing-Time.html (accessed December 16, 2014).

SSA. 2014i. Open government initiative. Data on combined Title II disability and Title XVI blind/disabled average processing time (in days) (excludes technical denials). http://www.ssa.gov/open/data/Combined-Disability-Processing-Time.html (accessed December 16, 2014).

SSA. 2014j. SSDI awards by diagnostic group and age of awardee under the age of 65, 2013 (preliminary data). Data submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 21, 2014.

SSA. 2014k. SSI annual statistical report, 2013. Washington, DC: SSA. http://www.ssa.gov/policy/docs/statcomps/ssi_asr (accessed February 24, 2015).

SSA. 2014l. SSI awards by diagnostic group and age of awardee under the age of 65, 2013. Data prepared by ORDP, ODP, and ODPMI. Data submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 21, 2014.

SSA. 2014m. Substantial gainful activity. http://www.socialsecurity.gov/oact/cola/sga.html (accessed December 15, 2014).

SSA. 2015. Hearings and appeals. ALJ disposition data, fiscal year 2015 (for reporting purposes: 09/2/2014 through 01/20/2015). http://www.ssa.gov/appeals/DataSets/03_ALJ_Disposition_Data.html (accessed February 27, 2015).

SSA. n.d.-a. Disability evaluation under Social Security—Part II: Evidentiary requirements. http://www.ssa.gov/disability/professionals/bluebook/evidentiary.htm (accessed September 4, 2014).

SSA. n.d.-b. Disability evaluation under Social Security—Part III: Listing of impairments. http://www.ssa.gov/disability/professionals/bluebook/listing-impairments.htm (accessed October 3, 2014).

SSA. n.d.-c. Disability evaluation under Social Security—Part III: Listing of impairments—Adult listings (Part A). http://www.ssa.gov/disability/professionals/bluebook/12.00-MentalDisorders-Adult.htm (accesseed October 3, 2014).

SSA. n.d.-d. Disability evaluation under Social Security—Part III Listing of impairments—Childhood listings (Part B). http://www.ssa.gov/disability/professionals/bluebook/ChildhoodListings.htm (accessed October 7, 2014).

SSA. n.d.-e. Disability evaluation under Social Security—Part III: Listing of impairments—Adult listings (Part A)—section 12.00 mental disorders. http://www.ssa.gov/disability/professionals/bluebook/12.00-MentalDisorders-Adult.htm (accessed November 14, 2014).

SSA. n.d.-f. Disability evaluation under Social Security—Part III: Listing of impairments—Childhood listings (Part B)—section 112.00 mental disorders. http://www.ssa.gov/disability/professionals/bluebook/112.00-MentalDisorders-Childhood.htm (accessed October 3, 2014).

SSA. n.d.-g. Hearings and appeals. Federal court review process. http://www.socialsecurity.gov/appeals/court_process.html#a0=1 (accessed October 7, 2014).

SSA. n.d.-h. Hearings and appeals. Information about requesting review of an administrative law judge’s hearing decision. http://www.socialsecurity.gov/appeals/appeals_process.html#a0=2 (accessed October 7, 2014).

SSA. n.d.-i. Hearings and appeals. What you need to know to request a hearing before an administrative law judge. http://www.socialsecurity.gov/appeals/hearing_process.html#a0=4&sb=3 (accessed October 7, 2014).

SSA. n.d.-j. How we decide if you are disabled. Information we need about your work and education. http://www.ssa.gov/disability/step4and5.htm (accessed October 7, 2014).

SSA. n.d.-k. Medical/professional relations. Consultative examinations: A guide for health professionals. http://www.ssa.gov/disability/professionals/greenbook (accessed October 16, 2014).

SSA. n.d.-l. Occupational Information System project. http://www.ssa.gov/disabilityresearch/occupational_info_systems.html (accessed December 30, 2014).

SSA. n.d.-m. Selected data from Social Security’s disability program. http://www.ssa.gov/oact/STATS/dibStat.html (accessed January 27, 2015).

SSDRC (Social Security Disability and SSI [Social Security Insurance] Resource Center). n.d. Applying for disability: How long does it take to get Soical Security Disability or SSI benefits? http://www.ssdrc.com/disabilityquestions1-46.html (accessed December 15, 2014).

Strand, A. 2002. Social Security disability programs: Assessing the variation in allowance rates. ORES working paper series, no. 98. Washington, DC: Social Security Administration, Division of Policy Evaluation, Office of Research, Evaluation, and Statistics. http://socialsecurity.gov/policy/docs/workingpapers/wp98.pdf (accessed February 4, 2015).

Ward, T. A. 2014. Disability Determination Services panel discussion. Presentation to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations, August 11, 2014, Washington, DC.

Wedding, D., and D. Faust. 1989. Clinical judgement and decision making in neuropsychology. Archives of Clinical Neuropsychology 4(3):233-256.