Topics for Further Study
The charge to the steering committee was a narrow one within a complicated set of issues, and in the course of the workshop and the deliberations that led to this report a number of related topics that merit further attention emerged. This is by no means a comprehensive list of the topics that deserve further work, and the committee is well aware that these and other issues are already being explored in many contexts around the country. However, this report is intended in part as a spur to further discussion of unresolved issues surrounding admissions testing, and the committee notes a few of the questions about which further data and research would be particularly useful.
- Beyond the survey data cited in this report, what is known about the range of practices in admissions offices around the country? To what extent are test scores being used in inappropriate ways? Is it possible to obtain this information, given both institutions' reluctance to reveal the details of their practice, as well as practical constraints on the collection of detailed, comprehensive data?
- In the past decade, test preparation courses have proliferated and raised a number of issues. Some research has been done to determine the extent to which coaching can increase scores, but it is not definitive. An objective assessment of the effects of various coaching methods on scores is clearly needed. Are the numbers of coached students sufficient for there to have been a discernable effect on aggregate scores? If these courses have even a small effect on scores, they also raise important questions about test validity. How does the content of coaching programs relate to the subject domains the tests are designed to measure? Coaching also raises questions about fairness: such programs generally cost hundreds or thousands of dollars, and while some public school districts have offered coaching to their students, it is clearly not equally available to all.
- What is known about the relationship between varying uses of admissions test scores and K-12 education, particularly efforts to reform it? To what extent can uses of test scores be modified to offset the tendency of scores to reinforce the cumulative advantages and disadvantages that students are subject to in the K-12 years?
- Do the tests currently used measure constructs that are genuinely relevant to the academic programs for which they serve as screens? Has
- the content of the current tests been objectively evaluated in terms of current knowledge about human thinking and learning?
- To what extent are questions surrounding the use of admissions tests the same for graduate and professional school programs as they are for undergraduate institutions?
- What have been the effects of reliance on alternative means of selecting students? What alternative predictive tools have been used and what is known about them?
- What effects—on the composition of classes, on other aspects of the admissions process, on the tests themselves, for example—might be expected if a greater number of schools deemphasized or stopped requiring test scores?
- What measures could improve understanding of the benefits and limitations of admissions tests and reduce the risk of misuse of test scores? What steps could test makers, accreditation associations, higher education associations, and others take to ensure that sound test use policies are developed and followed?