A Pragmatic Future for NAEP Containing Costs and Updating Technologies (2022) / Chapter Skim
Currently Skimming:
Pages 66-71
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 66... ...
Even without reaching the level of coordinating the content of different assessments, the practical and political costs involved in achieving this level of coordination across separate assessment programs is likely to be overwhelming. RECOMMENDATION 6-4: Efforts to coordinate NAEP test admin istration with the international assessment programs sponsored by the National Center for Education Statistics should not be used as a strategy to reduce costs.
|
From page 67... ...
The scoring and dissemination contract, also referred to as the materials, distribution, processing, and scoring contract, includes the following activities: "Prepares and packages all assessment and auxiliary materials; distributes assessment booklets and materials to the test administrators for each school; receives the materials from the schools; with [item development] and [design, analysis and reporting]
|
From page 68... ...
6 NCES response to Q69e. The activity to support assessment administration is described as follows: "State Service Center, State NAEP Coordinators, and State Testing Directors support." 7 "Automated scoring" and "machine scoring" are sometimes used as equivalent terms.
|
From page 69... ...
Importantly, the rapid improvements in recent years in computer algorithms and available data have the potential to further improve automated-scoring performance for existing 10 Human scoring performance is typically used as the standard for evaluating the perfor mance of automated scoring engines because it is the obvious alternative. 11 See also Gregg, N., Young, M., and Lottridge, S
|
From page 70... ...
. Consideration of these factors during item creation can result in items that can be scored more successfully by both humans and automated scoring engines.
|
From page 71... ...
Items in the mandated reading and mathematics assessments with state and urban district samples are at the top of the range of response counts: items used in assessments with national samples are at the bottom of this range. In most implementations, automated-scoring models are trained for every item, and so increasing items increases costs.15 The NAEP response counts per item are near the threshold for achieving cost savings from automated scoring, which is typically around 30,000 responses; it depends on the cost savings from hand scoring and the overall number of items automatically scored.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.