Skip to main content

Currently Skimming:

7 Simplifying NAEP's Technical Design: The Role of the Short Form
Pages 40-50

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 40...
... One or more versions of the short form would be released for public use, while other versions of the short form would be kept secure for use in conjunction with national NAEP and, perhaps, with state and local assessment programs. To guide policy and decision making on the measurement issues pertaining to the short form, NAGB adopted the following principles (National Assessment Governing Board, 1999a)
From page 41...
... At present, state assessment programs enable withinstate comparisons among schools and districts, but they do not allow for comparisons across state boundaries. State NAEP enables comparisons of achievement results from state to state but does not allow for comparisons among districts and schools, since results are not reported at the district and school levels.
From page 42...
... Embedding the Short Form in Existing Assessments Prior to the market-basket workshop, discussants were asked to consider the ways they would use the short form, if it were available. Many said that they would want to "embed" the short form in their state or district assessments to obtain results that could be compared with both the local assessments and with NAEP.
From page 43...
... I think that test directors are assuming that the proposed short form would be 10-15 items that could be used to scale the rest of the items on a NAEP scale, just as one embeds items on different levels of a test so that you can obtain a common scale across forms or grades. 43 O 'Reilly reported that respondents saw great value in obtaining normative data on NAEP-like tests but were adamant about incorporating items from a short form into existing tests.
From page 44...
... According to First in the World representatives, Paul Kimmelman and Dave Krooze, a key component of the consortium's efforts was the establishment of learning networks that allowed teachers and administrators to participate in the reform discussions and improvement efforts (Hawkes et al., 19971. The consortium established a research agenda covering four broad areas: student performance; curriculum and instruction; instructional practices; and teacher characteristics.
From page 45...
... If short forms were produced for subjects not tested as part of state and local assessments, then states and districts could use the short forms to expand their assessment programs. PROBLEMS WITH THE SHORT FORMS Scoring: Faster, Easier, Better' One advantage cited for the short forms was that they could be faster and less expensive to score than traditional NAEP assessments, providing score distributions without NAEP's usual complex statistical methodologies.
From page 46...
... In the words of one speaker, "Would it not be sufficient to provide results that are only somewhat NAEP-like? " Reliability and Generalizabilit~r The items selected for the short form are intended to represent NAEP's fourth grade mathematics frameworks.
From page 47...
... In preparation for the workshop, Patricia Kenney, co-director of the NCTM NAEP Interpretive Reports Project, considered the feasibility of creating market-basket forms that matched the grade four NAEP mathematics assessment on the basis of content strand coverage, ability category, and item type. Based on material in John Mazzeo's paper, Kenney reported that the market-basket forms appear to represent the frameworks in terms of the content strand.
From page 48...
... Under the current NAG B plan, two short forms would be produced, one for public release and the other kept secure and retained for use by states and districts. While school administrators might be able to control the generation of individual results from the secure form, the released form
From page 49...
... Some workshop discussants raised the question of how students' motivation to do well might factor into performance on the short form. State and local assessments tend to be high-stakes exams that carry consequences for those who do not perform well; thus, motivation to do well is high.
From page 50...
... Such uses of the NAEP items bring to the forefront issues about linking state and local assessments to NAEP. Several discussants referenced reports from two earlier NRC committees, the Committee on the Equivalency and Linkage of Educational Tests and the Committee on Embedding Common Test Items in State and District Assessments (National Research Council, 19994; 1 999a)


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.