Psychological Testing in the Service of Disability Determination (2015)

6

Economic Considerations

This chapter discusses the possible financial impact of the committee’s recommendations that the U.S. Social Security Administration (SSA) require systematic use of standardized psychological testing for a broader set of physical and mental impairments than is current practice for applicants who allege cognitive impairment or whose allegation of functional impairment is based solely on self-report. Although the committee’s recommendations are based on its assessment of the scientific evidence underlying standardized psychological testing and of the contributions such testing could make to determinations regarding the extent of impairment and degree of functional capacity in those populations, it recognizes that financial considerations also are relevant to decisions regarding implementation of psychological testing. In this context, the chapter provides an initial framework for evaluating the economic costs of implementation and highlights the types of data that will be needed to accurately determine the financial impact of mandatory psychological testing as recommended by the committee for disability determinations. A more thorough assessment of the financial implications is beyond the committee’s ability or charge.

The chapter begins with a discussion of the potential cost outlays associated with required psychological testing and describes how these costs vary by test type, provider, and geographical location. As a benchmark, simple cost estimates are provided, along with sensitivity analysis that illustrates the relationship between financial outlays and the size of the applicant population requiring testing. The chapter then focuses on the potential financial benefits of testing, primarily any cost savings from expanding the use of psychological testing as recommended by the committee. In this

context, the chapter discusses research arguing that requiring psychological testing, specifically symptom validity tests (SVTs) and performance validity tests (PVTs), will generate significant savings for the Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) programs by greatly reducing the number of “false” favorable determinations (false positives). The chapter concludes with a summary of the types of data that SSA and state Disability Determination Services (DDS) offices would need to collect in order to accurately assess the net financial impact of implementation.

COSTS OF PSYCHOLOGICAL TESTING

Costs of Psychological Testing Services

As the recommendations state, the administration of psychological testing would be part of the normal disability determination process. As such, applicants could provide any required tests in their initial application for disability benefits. In these cases, required psychological testing would impose no financial costs on SSA. For applicants without such tests, SSA could gather the information as part of case development. In some cases, testing may necessitate a consultative examination. In all cases, the costs to SSA of providing testing would relate to the administration and interpretation of all required tests.¹

To ensure that any test results are reliable, specialists appropriately trained in the administration and interpretation of standardized psychological tests would need to be used. Depending on the type of tests being given, trained providers include psychiatrists or other appropriately licensed physicians, licensed psychologists, and trained and licensed technicians.² One estimate of the current costs of these services comes from the Medicare reimbursement rates, which are updated yearly and are used to determine what Medicare will pay to providers treating Medicare patients. Table 6-1 reports average Medicare reimbursement rates in 2014 for psychological testing services provided outside of a facility such as a hospital.³ These services include

_____________

¹ It is difficult to project how many applicants would respond to testing requirements by seeking testing in advance of filing an application. One way SSA could estimate this is by examining the share of applicants with intellectual disabilities who file for benefits with all required testing in the application.

² In some cases tests, could be administered online using computer-administered tests. These tests still require a licensed provider to interpret the results.

³ In some cases, costs of services are significantly lower when provided inside a facility. Because most of the applicants for disability benefits live in the community rather than in an institution, the present discussion focuses on non-facility prices.

TABLE 6-1 Costs of Psychological and Neuropsychological Testing Services

^a Centers for Medicare & Medicaid Services provides pricing data for this code in 15-min-ute rather than hourly increments. Hence the data were transformed to hourly rates for the purpose of comparability to other codes.

SOURCE: CMS, 2015, and committee calculations.

(1) psychiatric diagnostic interview, HCPCS code 90791⁴; (2) psychological testing by a psychologist or physician, HCPCS code 96101; (3) psychological testing by a technician, HCPCS code 96102; (4) neurobehavioral status exam, HCPCS code 96116; (5) neuropsychological testing by a psychologist or physician, HCPCS code 96118; (6) neuropsychological testing by a technician, HCPCS code 96119; and (7) health and behavioral assessment, HCPCS code 96150. For purposes of comparison, the costs are shown for 1 hour of service. In practice, the time for evaluation varies with the type of testing required and the complexity of the case.⁵

The average cost of testing services varies by the type of testing, psychological versus neuropsychological, and by the type of provider, as in a psychologist or physician versus a technician.⁶ For an equivalent unit of service, a psychiatric diagnostic interview is the most expensive and was reimbursed by Medicare at an average rate of $134 in 2014. Psychological testing by a technician is the least expensive, with an average reimbursement rate of $66 in 2014.

As the minimum and maximum values in the table highlight, the cost of purchasing qualified psychological testing services of any type varies considerably across states and localities (SSAB, 2012, p. 52, Figure 47). For example, in the most expensive area, 1 hour of psychiatric evaluation costs $188 compared to $124 in the least expensive area. There is also substantial variation in service costs for general psychological testing, with the variation greater among technician-provided services than services provided by psychologists or physicians. The variation in pricing is similarly large for neuropsychological testing. For physicians or psychologists providing neuropsychological testing, Medicare reimbursement rates vary from $88

_____________

⁴ The codes listed reflect a sample of codes that may be used by providers.

⁵ The length of an evaluation will vary depending on the purpose of the evaluation, and more specifically, the type of psychological and/or cognitive impairments being assessed. Most psychological and neuropsychological evaluations include (1) a clinical interview, (2) administration of standardized cognitive or non-cognitive psychological tests, and (3) professional time for interpretation and integration of data. The relevant CPT codes for each of these processes are generally billed in 1 hour per unit of service (the exception is 96150, which is a 15 minute/unit code). That is, an evaluation may include billing for 1 hour for clinical interview (96116), 1 hour for administration of tests (96119), and 1 hour for interpretation and integration (96118) for a total of 3 hours of clinical service. However, a more complex case likely will require additional hours of test administration and interpretation/integration in order to fully answer the clinical question. In fact, the results of a national professional survey indicate that billing for a typical neuropsychological evaluation is roughly 6 hours, with a range from 0.5 to 25 hours (Sweet et al., 2011).

⁶ The table includes both weighted and unweighted averages. Weighted averages are appropriate for considering total costs to SSA since they are weighted to reflect population differences across counties in which the reimbursement rate holds. Unweighted averages provide information relevant to considering cost dispersion across states. Average prices referenced in the text reflect weighted averages.

to $134 per hour/unit billed depending on location. The variation is even larger for technicians as reflected in the larger standard deviation of reimbursement rates. In general, price variation occurs for all testing types with the exception of the health and behavioral assessment.

The variation in pricing of services by geographical area implies that the costs to SSA of requiring psychological testing will depend, in part, on the geographical location of the applicants most likely to require testing. As shown in Chapter 2, there is considerable variation in application filing rates for disability benefits across U.S. states. This variation suggests that the demand for psychological testing for disability determinations will also vary, resulting in larger outlays in some states than in others. Whether this variation in demand for testing services interacts with variation in testing prices to reduce or increase costs is something that would have to be investigated once testing is implemented.

Part of the service price variation shown in Table 6-1 owes to regional differences in overall price levels. However, differences in the availability of providers and the overall demand for psychological services in the area may also play a role. In markets where providers are limited but filing rates for SSDI or SSI are high, required use of psychological testing by SSA potentially could increase demand for testing services sufficiently to have an impact on service prices. Given the small share of disability applicants relative to the population, this seems unlikely in large metropolitan areas. However, in smaller rural areas or states with fewer providers, any increase in demand for services might affect market prices. To the extent that testing could be computer administered and scored and interpreted by a provider living outside of the applicant’s geographical area, these impacts would be lessened. Determining the best method to provide testing services cost-effectively to disability benefit applicants would be an important element of implementing the recommendations in this report.

Another factor that could push up costs relative to the numbers in Table 6-1 is that providers may demand higher payments than those offered by Medicare. DDS offices are not under the Medicare reimbursement rules, and if providers asked for more to provide required psychological services presumably the offices and SSA would have to pay those rates. Finally, it is possible that the use of psychological testing by SSA could create a market for test preparation or test coaching that would in turn lead to a need for new and improved tests, and then more coaching, and so forth. Should this occur, the costs of testing by SSA could potentially rise over time. The likelihood of this type of “testing spiral” and its impact on costs is something that could be monitored and assessed in the early stages of implementation.

There are also potential cost offsets that might make testing less expensive for SSA than the Medicare reimbursement rates would suggest. For example, if SSA decides to use testing on a large scale it might be able to

purchase licenses for testing products or contract with a national provider of testing services, resulting in lower fees for service. With respect to geographic considerations, SSA might be able to rely on telemedicine for clinical interviews and/or technician administration of tests, with offsite interpretation by psychologist/neuropsychologists on large national or regional contracts. SSA could consult with the Veterans Health Administration or private disability insurers to assess the feasibility and likely cost savings of these alternatives.

Tested Populations and Estimates of Costs

The cost of requiring psychological testing depends on the price of the tests and on the number of individuals who must be tested. There is no straightforward way to map the committee’s recommendations regarding who should receive psychological testing onto SSA’s publicly available data to derive an accurate measure of the size of the tested population.⁷ However, the data do permit the calculation of cost estimates associated with testing groups of applicants the committee judges to be most likely to fall under the recommendations in this report. The results of this exercise are provided in Table 6-2. The table shows cost computations for testing applicants who reach Step 4 or 5 of the disability determination process described in Chapter 2. These are individuals who did not qualify for benefits by meeting or equaling the medical listings but were sent along for further evaluation, rather than being denied. By definition, these are individuals for whom a determination regarding benefits requires further case development, including assessment of their ability to perform substantial gainful activity at some job in the national economy.⁸ In addition to calculations for all applicants reaching this stage, the table shows cost estimates should psychological testing be required for the subset of applicants with mental impairments other than intellectual disabilities or arthritis and back disorders.

The results from this exercise demonstrate the variation in projected costs associated with factors related to implementation including which tests will be required, the qualifications mandated for testing providers, and the number of individuals who will need to be tested. For example, if SSA provided psychiatric diagnostic interviews at the average Medicare reimbursement rate for all applicants reaching Step 4 or 5, the cost would

_____________

⁷ SSA collects a variety of data that it does not provide publicly and may be able to do a more accurate initial assessment of the costs associated with the recommendations. However, to fully measure the potential costs it is likely that SSA would need to pilot the use of testing and the costs associated with it.

⁸ For children applying for SSI, the evaluation is based on attending school rather than working.

TABLE 6-2 Estimated Costs of Testing

NOTE: Based on 2013 application data and 2014 Medicare pricing information, geographically weighted. Values in Table 6-2 may not exactly reflect multiplication of weighted pricing data from Table 6-1 and number of persons in column one of Table 6-2 due to rounding error.

SOURCES: CMS, 2015; SSA, 2014c,d,e; and committee calculations.

be $212 million. This cost would drop to $51 million if such testing were only provided to applicants with mental disorders (excluding intellectual disabilities). Similarly, costs would be lower if other forms of psychological testing were required or if other types of service providers were used.

Importantly, the cost estimates in Table 6-2 assume that SSA will be responsible for all the costs of psychological testing. However, as noted previously, some applicants may acquire and include required tests as part of the medical records presented at application. In this case, the cost to SSA would be minimal, providing that the disability determination offices already have sufficient personnel to adequately evaluate the test findings.

Another assumption implicit in this simple cost calculation is that the psychological testing would be added to current DDS case development costs. To the extent that psychological testing replaces rather than augments existing case development modalities, the costs to SSA would be lower than the simple estimates in the table. There are good reasons to believe that this might be the case. Consultative exams are already a common component of disability determinations.⁹ Some of these exams include psychological testing and it might be possible to add additional tests with limited additional costs.

Of course, the estimates in Table 6-2 could also understate the costs, especially since the calculations rely on a mapping of the recommendations to publically available data that may insufficiently capture the true number of individuals who could require testing. Accurately assessing the costs of mandatory psychological testing by SSA will require more detailed information on the parameters of implementation as well as experience in the field once testing has begun.

ASSESSING THE BENEFITS OF PSYCHOLOGICAL TESTING

Recent calls for greater use of psychological testing in SSA’s disability determination process assume that the current process is making significant mistakes and allowing unqualified applicants onto the disability programs (Chafetz and Underhill, 2013; IOPC, 2013). However, the committee has been unable to uncover any evidence on either side of this claim. At present, there do not appear to be any independently conducted studies regarding the accuracy of the disability determination process as implemented by DDS offices. As such, it is difficult to assess whether greater use of psychological testing will increase, decrease, or leave unchanged the number of individuals awarded benefits. The outcome depends on how accurately DDS offices currently are in making disability determinations.

_____________

⁹ On average 47 percent of disability evaluations include a consultative examination, although there is considerable variation across states (SSA, 2014a,b).

Even if DDS offices are making relatively accurate determinations in the absence of psychological testing, greater standardization could produce other benefits. A more standardized process could potentially reduce the number of applicants who appeal their decisions. For applicants who do appeal, the inclusion of psychological testing in the medical records could help reduce the burden on administrative law judges to make subjective determinations on the adequacy of the claim. Standardization might also make the process more transparent and efficient, improving public understanding and reducing the time it takes to process claims. However, none of these potential benefits can be quantified without additional research on the accuracy and efficiency of current practice. Such an assessment is an important first step in developing an implementation strategy for the committee’s recommendations.

ESTIMATES OF COST SAVINGS FROM PSYCHOLOGICAL TESTING

One of the main purported benefits of mandatory psychological testing is its potential to generate significant savings for the SSDI and SSI programs. The proponents of this view argue that requiring psychological testing (SVTs and PVTs) for SSDI and SSI applicants would result in a significant reduction of the number of individuals allowed onto the benefit rolls. For example, Chafetz and Underhill (2013) estimate that requiring SVTs and PVTs in the DDS process would save approximately $12.8 billion for the SSDI system and $7.2 billion for the SSI system, or about 40 percent of total program costs (see Tables 6-3 and 6-4, reproduced from Chafetz and Underhill [2013]). The estimated savings results from the assumed reduction in the number of falsely awarded individuals coming onto the disability programs.¹⁰

The committee performed a critical evaluation of this estimate and concluded that it is based on several assumptions that if violated would substantially lower the projected cost savings. Most important is the assumption that the current disability determination process, as implemented by DDS offices, is unable to detect any applicants who exaggerate or fabricate their impairments and related functional limitations. Although not stated directly in the analysis, this assumption is implicit in the authors’ use of base rates of malingering from populations of applicants and claimants ex ante of any disability screening. For example, the $12.8 and $7.2 billion savings computed by Chafetz and Underhill (2013) assumes that 40 percent of current SSDI

_____________

¹⁰ Improved accuracy could also decrease the number of individuals falsely denied benefits. However, the focus of the literature has been on reducing those falsely allowed onto the program.

TABLE 6-3 Calculation of 2011 SSDI Costs for Each Level of Malingering of Mental Disorders

NOTES: The 40 percent rate is bolded as the probable rate of malingering given in Larrabee, Millis, and Meyers (2009). For the SSDI total, the number of disabled workers is used, removing spouse and child beneficiaries. Costs were estimated by multiplying the average disability figure for each mental condition by the December 2011 number of individuals with that condition, summing over all conditions, and then multiplying by 12 for the yearly estimated amount. B = billion.

SOURCE: Chafetz and Underhill, 2013. Reproduced with permission.

TABLE 6-4 Calculation of 2011 SSI (Adult) Costs for Each Level of Malingering of Mental Disorders

NOTES: The 40 percent rate is bolded as the probable rate of malingering given in Larrabee, Millis, and Meyers (2009). The SSI figures include the number of adults (less than age 65) minus the children as of December 2011. Costs were estimated by multiplying the average disability figure for each mental condition by the December 2011 number of individuals with that condition, summing over all conditions, and then multiplying by 12 for the yearly estimated amount. B = billion.

SOURCE: Chafetz and Underhill, 2013. Reproduced with permission.

and SSI beneficiaries were falsely awarded and would have been denied benefits if given a SVT or PVT as part of the disability determination process. This assumption is synonymous with the view that DDS offices currently detect no one who exaggerates or fabricates their condition, symptoms, or functional limitations. In other words, the Chafetz and Underhill computation assumes that under current practice 40 percent of all awardees are given benefits even though they are not truly eligible. The extremeness of the Chafetz and Underhill assumption suggests that the cost savings associated with psychological testing is likely to be lower than they suggest.

The other important assumption embedded in the Chafetz and Underhill projected cost savings is that SVTs and PVTs would be retroactively applied to the population of existing beneficiaries, regardless of time on the program.¹¹ Should SSA choose to implement mandatory SVT and PVT testing, it would likely do so for new applicants to the disability programs, making the potential cost savings lower than that computed by Chafetz and Underhill.

Finally, the Chafetz and Underhill calculation is static. The more appropriate method of computing cost savings is to consider the present discounted value of an estimated stream of potential benefit savings, which would generate a much larger estimate.

The importance of altering the assumptions about improved accuracy of disability determinations and the size of the population exposed to testing can be seen in Table 6-5. Reflecting the mapping of the committee’s recommendations for testing used in Table 6-2, cost savings are estimated for new awardees with mental impairments other than intellectual disabilities and for those with arthritis and back disorders. For completeness, the estimates are also provided for all new beneficiaries, regardless of condition and for all awardees and awardees determined eligible in Steps 4 or 5 of the disability determination process. The alternative estimates also show the sensitivity of the estimated cost savings to the assumption about the potential for mandatory SVT and PVT use to improve the accuracy of SSA disability determinations. The 40 percent test failure rate preferred by Chafetz and Underhill (2013) applies if the current SSA process detects zero percent of those who exaggerate or fabricate; the 10 percent test failure rate applies if SSA is relatively accurate, but makes some false-positive errors that would be identified through the use of SVTs and PVTs.

Several important points emerge from the computations in the table. First, the potential annual cost savings associated with mandatory SVT and PVT testing is substantially reduced when it is applied to new awardees

_____________

¹¹ Chafetz and Underhill (2013) limit the group to those with mental disorders, but even so this assumption greatly increases the cost savings associated with greater use of testing, because it essentially applies the 40 percent base malingering rate to all existing beneficiaries.

TABLE 6-5 Estimated Annual Savings of Testing New Disability Awardees

^a SSDI benefit data are from 2012, and SSI and concurrent benefit data are from 2013. For concurrent enrollees, there are no data available on average benefit payments by diagnosis, so the average benefit level for all persons was used for all concurrent enrollment calculations. For SSDI and SSI, the average benefit amount for mental disabilities (excluding intellectual disability) was calculated as a weighted average of the average monthly benefits awarded for mental disability diagnoses (excluding intellectual disability) using diagnostic distribution data. For musculoskeletal conditions, there are no data available specifically for back disorders and arthritis, so the average benefit for musculoskeletal disorders was used to calculate estimated savings. SSA did not have information concerning average SSI benefits by diagnosis available separately for children and adults, so a single weighted average was used for both groups using diagnostic and benefit distributions for all recipients under age 65.

^b SSDI diagnostic distribution data are from 2012. SSI and concurrent enrolled diagnostic distribution data are from 2013.

^c All disability application data are from 2013.

^d Test failure rates are synonymous with what some literature refers to as malingering rates.

SOURCES: SSA, 2014c,d,e, and committee calculations.

rather than all beneficiaries on the programs. Considering only new awardees with mental impairments other than intellectual disabilities, the cost savings assuming the 40 percent malingering rate is $236 million for SSDI and $153 million for SSI, about one-fifth of the savings reported by Chafetz and Underhill (2013). Second, cost savings are also reduced when the assumption about the accuracy improvements associated with symptom and validity testing are relaxed. If SSA misses 10, rather than 40, percent of those with exaggerated or fabricated claims, the cost savings from mandatory testing on new awardees with mental impairments other than intellectual disabilities falls from $236 to $59 million for SSDI and from $153 to $38 million for SSI adults. Finally, cost savings decline if testing is required only for applicants who reach Steps 4 or 5 of the disability determination process. Although these estimates are far from exact, they suggest that caution is warranted when projecting potential cost savings from mandatory psychological testing.

As noted earlier, the static calculations in Table 6-5, although useful for comparing to Chafetz and Underhill, are not appropriate for computing the expected savings associated with implementing SVTs and PVTs in SSA’s disability determination process. The expected program savings is more accurately calculated as the present discounted value of the averted payment flows associated with the denied applicants captured by psychological testing. Using the same diagnostic categories as in Table 6-5, Table 6-6 shows the present discounted value of expected savings from disallowing an unqualified applicant from each of the three disability programs. The table also shows the estimated program savings to SSA under the assumption that psychological testing as recommended would result in the denial of benefits to 10 percent of applicants who would otherwise receive them.

Two points emerge from the table. First, the expected cost savings associated with denying an applicant improperly allowed on the program can be sizeable, depending on the diagnosis and program. The estimated savings are largest for individuals with mental impairments; this reflects the earlier age of benefit receipt and longer average time on the program. Estimated savings are smallest for SSI recipients with arthritis and back pain, again largely reflecting the age at which recipients enter the program. Second, the amount of program savings that comes from implementing psychological testing depends mostly on how many additional individuals would be identified as unqualified for benefits relative to current practice. It is important to keep in mind that psychological testing as recommended may also result in the awarding of benefits to some portion of applicants who otherwise would be denied. Assuming that implementation of psychological testing reduces the number of newly awarded beneficiaries by 10 percent, the savings per cohort, while significant, still would be less than the annual savings estimated by Chafetz and Underhill.

TABLE 6-6 Estimated Lifetime Spending on an Individual Disability Awardee, 2 Percent Annual Discounting

NOTE: SSDI benefit data from 2012, SSI from 2013. The average benefit amount for mental disabilities (excluding intellectual disability) was calculated as a weighted average of the average monthly benefits awarded for mental disability diagnoses (excluding intellectual disabilities) using diagnostic distribution data. For musculoskeletal conditions, there are no data available specifically for back disorders or arthritis, so the average benefit for musculoskeletal disorders was used to calculate estimated savings. Overall average benefit by program was used to calculate “all diagnostic groups” savings. SSA did not have information concerning average SSI benefits by diagnosis available separately for children and adults, so a single weighted average was used for both groups using diagnostic and benefit distributions for all recipients under age 65. Average time spent on disability benefits by diagnosis comes from Riley and Rupp (2014, Table 3). As Riley and Rupp do not differentiate between programs, the same value was used for all programs within a diagnosis.

FINDINGS

Understanding the financial costs and benefits of using psychological testing in the SSA disability determination process is an important, but unfinished, task. The data necessary to make accurate calculations are limited, and estimates based on available data are subject to considerable error. That said, the framework for a proper computation is well understood and can be used to guide data collection and evaluation when testing is and is not employed.

Accurate assessments of the net financial impact of mandatory psychological testing will require information on the current accuracy of DDS decisions and how the accuracy is improved, or unaffected, by the use of more

standardized testing. It will also be important to determine which types of tests should be given and to which groups in the applicant population. This information can then be used to consider the impact on the demand for testing services across the country and whether or not that demand affects service pricing. All of these components could be gathered in pilot programs that allow for experimentation and assessment prior to wider implementation. In addition, the committee found:

• The average cost of testing services varies by the type of testing (e.g., psychological, neuropsychological), by the type of provider (e.g., psychologist or physician, technician), and by geographical area. The variation in pricing implies that the expected costs to SSA of requiring psychological testing will depend on exactly which tests are required, the qualifications mandated for testing providers, and the geographical location of the providers most in demand.
• Estimating the exact cost of broad use of psychological testing by SSA will require more detailed data on the exact implementation strategy. To fully measure the potential costs, it is likely that SSA will need to pilot the use of testing and the costs associated with it.
• Some published estimates of the potential cost savings to SSA associated with the use of symptom validity testing and performance validity testing are based on assumptions that if violated would substantially lower the estimated cost savings. Potential cost savings associated with testing vary considerably based on the assumptions about who it is applied to and how many individuals it detects and thus rejects for disability benefits.
• At present, there do not appear to be any independently conducted studies regarding the accuracy of the disability determination process as implemented by DDS offices.
• A full financial cost-benefit analysis of psychological testing will require SSA to collect additional data both before and after the implementation of the recommendations of this report.

REFERENCES

Chafetz, M., and J. Underhill. 2013. Estimated costs of malingered disability. Archives of Clinical Neuropsychology 28(7):633-639.

CMS (Centers for Medicare & Medicaid Services). 2015. Physician fee schedule search tool. http://www.cms.gov/apps/physician-fee-schedule/search/search-criteria.aspx (accessed January 20, 2015).

IOPC (Inter Organizational Practice Committee). 2013. Use of symptom validity indicators in SSA psychological and neuropsychological evaluations. Letter to Senator Tom Coburn. https://www.nanonline.org/docs/PAIC/PDFs/SSA%20and%20Symptom%20Validity%20Tests%20-%20IOPC%20letter%20to%20Sen%20Coburn%20-%202-11-13.pdf (accessed February 8, 2015).

Larrabee, G. J., S. R. Millis, and J. E. Meyers. 2009. 40 plus or minus 10, a new magical number: Reply to Russell. The Clinical Neuropsychologist 23(5):841-849.

Riley, G. F., and K. Rupp. 2014. Cumulative expenditures under the DI, SSI, Medicare, and Medicaid programs for a cohort of disabled working-age adults. Health Services Research 50(2):514-536. doi: 10.1111/1475-6773.12219.

SSA (Social Security Administration). 2014a. DDS performance management report. Disability claims data. Consultative examination rates, fiscal year 2013. Data prepared by ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on August 25, 2014.

SSA. 2014b. Disability claims data (initial, reconsideration, continuing, disability review) by adjudicative level and body system. SSDI, SSI, concurrent, and total claims allowance rates for claims with consultative examinations by U.S. states, fiscal year 2013. Data prepared by ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on August 25, 2014.

SSA. 2014c. National data Title II-SSDI, Title XVI-SSI, & Concurrent Title II/XVI initial disability determinations by regulation basis code (reason for decision), fiscal year 2013. All cases except mental disorders (other than intellectual disability) and arthritis and back diorders. Data prepared by SSA, ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 23, 2014.

SSA. 2014d. National data Title II-SSDI, Title XVI-SSI, & Concurrent Title II/XVI initial disability determinations by regulation basis code (reason for decision), fiscal year 2013. Arthritis and back disorders only. Data prepared by SSA, ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 23, 2014.

SSA. 2014e. National data Title II-SDI, Title XVI-SSI, & Concurrent Title II/XVI initial disability determinations by regulation basis code (reason for decision), fiscal year 2013. Mental disorders only (excluding intellectual disability). Data prepared by SSA, ORDP, ODP, and ODPMI. Submitted to the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations by Joanna Firmin, Social Security Administration, on October 23, 2014.

SSAB (Social Security Advisory Board). 2012. Aspects of disability decision making: Data and materials. Washington, DC: SSAB.

Sweet, J. J., D. G. Meyer, N. W. Nelson, and P. J. Moberg. 2011. The TCN/AACN 2010 “salary survey”: Professional practices, beliefs, and incomes of U.S. neuropsychologists. The Clinical Neuropsychologist 25(1):12-61.

This page intentionally left blank.