Skip to main content

A Pragmatic Future for NAEP Containing Costs and Updating Technologies (2022) / Chapter Skim
Currently Skimming:

3 Possible Structural Changes
Pages 27-36

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 27...
... However, the panel did not consider the options of simply eliminating subjects or reducing the frequency of assessments as cost saving measures. The statement of task from the Institute of Education Sciences urged the panel to suggest options that would save money without impinging on the valuable information NAEP currently provides to its policy makers and the public.
Read the entire page →
From page 28...
... Keeping assessments fixed cannot guarantee trend maintenance when so much else is changing. Maintaining two programs within NAEP for tracking trends in reading and mathematics achievement is expensive, although the long-term trend assessments are relatively cheap because they use only national-level samples and typically have no costs for item development.
Read the entire page →
From page 29...
... The LTT assessments will become increasingly irrelevant as students perform greater amounts of their reading online, and reading assessments move into the digital age." Furthermore, in mathematics, she noted: "[T] he LTTs emphasize knowledge and skill much more than problem solving, making them essentially basic skills assessments, with some of the content outdated." See https://www.nagb.gov/ content/dam/nagb/en/documents/newsroom/naep-releases/naep-long-term-trend-symposium/ Content%20of%20LTT%20Compared%20to%20Main%20NAEP_Ina%20Mullis%20 021317_FINAL.pdf.
Read the entire page →
From page 30...
... Other ideas were offered at the 2017 symposium.3 If pursued, this effort would need to include a bridge study for transition to a digitally based assessment to minimize cost and increase relevance, as Mullis, Kolstad, and Haertel suggested in the NAGB symposium.4 In addition, it would be wise to undertake a renaming effort to minimize ongoing confusion between long-term trend and main NAEP and the trend information they provide. RECOMMENDATION 3-1: The National Center for Education Sta tistics should prepare a detailed plan and budget for the modernization of long-term trend NAEP, including the costs of creating post-hoc as sessment frameworks, bridging between paper and digital assessment, maintaining trends, and ongoing costs after the bridge.
Read the entire page →
From page 31...
... In addition to helping ensure the maintenance of trend lines for main NAEP, the use of standing framework committees to update NAEP's frameworks could also have some cost implications, both by lowering costs associated with protecting trends when proposed framework updates are drastic and by potentially using the existing subject-matter committees to update the frameworks rather than appointing standalone framework update committees. This change would require some institutional innovation -- and close collaboration between NAGB and NCES -- but the benefit for protecting NAEP trend data could be substantial.
Read the entire page →
From page 32...
... Our statement of task asked us to consider potential cost savings related to "substantive overlaps between NAEP assessments"; the possibility of combining assessments in complementary subject areas is the second way the panel considered interpreting that request, after considering the overlapping trend information in reading and mathematics. The panel considered several subject pairings.
Read the entire page →
From page 33...
... NAEP has traditionally assessed all four as separate assessments, though the current assessment schedule shows no plans to assess economics and geography through 2030.11 Two other potential subject groupings are not reflected in current state assessment practice as combined assessments but involve substantive relationships across assessments that may be meaningful to reflect in NAEP: reading with science or history, and mathematics and science. NAEP's new reading framework (NAGB, 2021)
Read the entire page →
From page 34...
... With respect to potential cost savings from combining assessments the primary opportunities lie with the three subject combinations -- language arts, science and engineering, and social studies -- that are already reflected in current state assessments. There are several relevant considerations with respect to the net benefit of such combinations: need for new frameworks, assessment schedule, sample size, and preserving subjects.
Read the entire page →
From page 35...
... Although there would be upfront investment costs to develop combined assessments, they could result in cost savings from reducing the number of assessments. The cost savings are likely to be small in most cases because at least one of the assessments in each pairing is given infrequently and usually to only a national sample.
Read the entire page →

Key Terms

  • main naep
  • assessment schedule
  • framework updates
  • trend data
  • trend information

  • reading framework
  • national assessment
  • potential cost
  • social studies
  • engineering literacy

  • combined assessments
  • separate assessments
  • item authors
  • language arts
  • trend line

  • naep reading
  • trend lines
  • state assessment
  • assessment governing
  • naep measures

  • naep validity
  • framework committees
  • frequent framework
  • reading assessments
  • integrated assessment

  • studies panel
  • validity studies
  • content experts
  • engineering practices
  • separate subject


    • This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
    More information on Chapter Skim is available.