Equivalency and Linkage of Educational Tests Interim Report (1998) / Chapter Skim
Currently Skimming:
Findings
Findings
Pages 12-31
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 12... ...
Offthe~shelf commercial tests anc3 tests that are custom clevelopec3 for states are increasingly constructed as mixec3-mociel assessments that contain different types of items, including multiple~choice items anc3 various kinds of open~enciec3 questions for which students construct their own responses by filling in a blank, solving a problem, writing a short answer, writing a longer response, or completing a graph or diagram (see, e.g., Shavelson, Baxter, and Pine, 1992~; Coloraclo, Connecticut, North Carolina, and Maryland are examples of states with mixec3-moclel assessments. Some item types are very useful for testing student recall of factual material (a claim often macle for certain types of multiple choice items)
|
From page 13... ...
They are the Maryland School Performance Assessment Program (MSPAP) , a performance assessment used in high-stake school evaluations and the Comprehensive Test of Basic Skills, Fourth Edition or CTBS/4, published by CTB/McMillan McGraw-Hill in 1989.
|
From page 14... ...
This is true because NAEP, by intent, floes not produce scores for inclivicluals anc3 because incliviclual students complete different parts of an entire NAEP assessment. STABILITY OF RESULTS The testing landscape in the United States is not only diverse, but it is dynamic: states anc3 districts have moved rapidly, especially cluring the last 10 years, to adopt new educational goals, new models of testing anc3 assessment, anc3 new strategies for aligning tests anc3 assessments to state content stanciarcis (National Research Council, ~ 997 )
|
From page 15... ...
. TEST USES AND EFFECTS ON TEACHER AND STUDENT BEHAVIOR Many states use assessments for multiple purposes related to educational improvement, such as program evaluation, curriculum planning, school performance reporting, anc3 student diagnosis (U.S.
|
From page 16... ...
Credit by Examination Direct Writing Assessment, Direct Mathematics Assessment Iowa Test of Basic Skills, Test of Achievement Proficiency Stanford Achievement Test 8 Iowa Test of Basic Skills Form K, Test of Achievement Proficiency Custom developed Custom developed No mandated statewide testing program, approximately 99 percent of all districts participate in the Iowa Test of Basic Skills on a voluntary basis Custom developed Custom developed California Achievement Test 5 Custom developed Illinois Goals Assessment Program Indiana StatewideTesting for Educational Progress Plus Kansas Assessment Program (Kansas University Center for Educational Testing and Evaluation) Kentucky Instructional Results Information System Louisiana Educational Assessment Program Maine Educational Assessment (Advanced Systems in Measurement Inc.)
|
From page 17... ...
Grade 11 High School Proficiency Test, Grade 8 Early Warning Test New Mexico High School Competency Exam, Portfolio Writing Assessment, Reading Assessment for Grades 1 and2 Occupational Education Proficiency Examinations, Preliminary Comptency Tests, Program Evaluation Tests, Pupil Evaluation Program Tests, Regents Competency Tests, Regents Examination Program, Second Language Proficiency Examinations North Carolina End of Grade Fourth-, Sixth-, Ninth-, and Twelfth-Grade Proficiency Tests Oklahoma Core Curriculum Tests Reading, Writing, and Mathematics Assessment Writing, Reading, and Mathematics Assessment Health Performance Assessment, Mathematics Performance Assessment, Writing Performance Assessment Basic Skills Assessment Program Stanford Achievement Test 9, Custom developed Has a voluntary state assessment program Customized off the shelf Tennessee Comprehensive Assessment Program (TCAP) Achievement Test Grades 2-8, TCAP Competency Graduation Test, TCAP Writing Assessment Grades 4,8, and 11.
|
From page 18... ...
Customized off-the-shelf assessments result from modifications to a commerical test publisher's existing product. SOURCE: Data from 1997 Council of Chief State School Officers Fall State Student Assessment Program Survey 19981.
|
From page 19... ...
FINDINGS TABLE 2 Student Testing: Diversity of Purpose 19 State Decisions for Students Decisions for Schools Instructional Purposes Alabama High school graduation School performance Student diagnosis or placement; reporting Improve instruction; Program evaluation Alaska School performance Improve instruction reporting Arizona School performance Student diagnosis or placement; reporting Improve instruction; Program evaluation Arkansas School performance Student diagnosis or placement; reporting Improve instruction; Program evaluation California Student diagnosis or placement Student diagnosis or placement Coloradoa Connecticut Student diagnosis or Awards or recognition; Student diagnosis or placement; placement School performance Improve instruction; reporting Program evaluation Delaware Student diagnosis or placement; Improve instruction; Program evaluation Florida High school graduation Improve instruction; Program evaluation Georgia High school graduation School performance Student diagnosis or placement; reporting Improve instruction; Program evaluation Hawaii High school graduation Awards or recognition; Student diagnosis or placement; School performance Improve instruction; reporting Program evaluation Idaho School performance Improve instruction reporting Iowaa Illinois Accreditation Indiana Awards or recognition; Student diagnosis or placement; School performance; Improve instruction; reporting Program evaluation Kansas School performance; Student diagnosis or placement; reporting; Accreditation Improve instruction; Program evaluation Kentucky Awards or recognition Improve instruction; Program evaluation Louisiana Student Promotion; Awards or recognition; Student diagnosis or placement; High school graduation School performance Improve instruction; reporting Program evaluation Maine Student diagnosis or Improve instruction; placement Program evaluation cornered
|
From page 20... ...
20 TABLE 2 (Continued) EQUIVALENCY AND LINKAGE OF EDUCATIONAL TESTS State Decisions for Students Decisions for Schools Instructional Purposes Maryland High school graduation School performance Student diagnosis or placement; reporting; Skills guarantee; Improve instruction; AccreditationProgram evaluation Massachusetts School performanceImprove instruction reporting Michigan Student diagnosis or Awards or recognition;Improve instruction; placement; Endorsed School performanceProgram evaluation Diploma reporting; Accreditation Minnesotan Mississippi High school graduation School performanceStudent diagnosis or placement; reporting; Skills guarantee; Improve instruction; AccreditationProgram evaluation Missouri School performanceImprove instruction; reporting; AccreditationProgram evaluation Montana Improve instruction; Program evaluation Nebraskan Nevada High school graduation School performanceImprove instruction; reporting; AccreditationProgram evaluation New Hampshire Improve instruction; Program evaluation New Jersey High school graduation School performanceStudent diagnosis or placement; reporting; AccreditationImprove instruction New Mexico High school graduation School performanceStudent diagnosis or placement; reporting; AccreditationImprove instruction; Program evaluation New York Student diagnosis or School performanceImprove instruction; placement; Student reportingProgram evaluation Promotion; Honors diploma; Endorsed diploma; High school graduation North Carolina Student diagnosis or Improve instruction; placement; Student Program evaluation Promotion; High school graduation North Dakota Student diagnosis or Student diagnosis or placement; placement Improve instruction; Program evaluation Ohio High school graduation Awards or recognition;Improve instruction; School performanceProgram evaluation reporting Oklahoma School performanceStudent diagnosis or placement; reporting; AccreditationImprove instruction; Program evaluation
|
From page 21... ...
SOURCE: Data from 1996 Council of Chief State School Officers Fall State Student Assessment Program Survey
|
From page 22... ...
These problems are attributable in part to the tests themselves, but linkage magnifies them anc3 increases the risk of unfair inferences about incliviclual achievement. REPORTING RESULTS IN TERMS OF THE NAEP ACHIEVEMENT LEVELS Linking other tests to NAEP raises the possibility of reporting incliviclual student scores on state anc3 commercial tests in terms of the NAEP achievement levels.
|
From page 23... ...
FINDINGS 23 assessment scores and NAEP scores; to potential differences in linking functions by different subgroups; and to other unicientifiec3 sources of measurement error. The committee has not been able to conduct a thorough study of parental and public reaction to this kind of scenario, but we caution that one of the more important putative purposes of linkageprovicling clear and relevant information about the performance of incliviclual students might be severely uncierminec3 by the need to report information which, in order to be faithful to the underlying statistics, must be ambiguous in its meaning.
|
From page 24... ...
The linkages had to be adjusted separately for different ethnic groups demonstrating that the linking was inappropriate for predicting individual scores from the North Carolina Test to the NAEP scale. The following were considered important factors in establishing a strong link: content on the North Carolina Test was closely aligned with state curriculum and NAEP's was not; student performance was effected by the order of the items in their test booklets; motivation or fatigue effects performance for some students.
|
From page 25... ...
, the moderate assessment programs Achievement Test for their state content alignment, the with the NAEP testing program, state results motivation of the students, or results for those had to be converted to a the nature of the student states. common scale.
|
From page 26... ...
students meet the common mean and standard Results of the linking were NAGB mathematics deviation on the two tests. different for countries with standards Translated IAEP scores into high average IAEP scores NAEP scores by aligning the Different methods of linking means and standard deviations the IAEP and NAEP can for the two tests produce different results, and Transformed the IAEP scores for further study is necessary to students in the IAEP samples in determine which method is each participating country into best equivalent NAEP scores Linking to a To compare the A sample of 8,239 applicants for Statistically, an accurate Large-Scale mathematics military service were distribution of recruit Assessment: achievement of new administered an operational achievement can be found by An Empirical military recruits ASVAB and a NAEP survey in projecting onto the NAEP scale Evaluation with the general 1992.
|
From page 27... ...
Connecticut study. It was possible to develop a Mastery Test, a For a relevant population, link between the MAT 6 and criterion-referenced calibrated the items from the the Connecticut Mastery test closely aligned two instruments in a given Test that accurately with state subject as a single IRT predicted Normal Curve curriculum, and a calibration then used the Equivalent scores for the national "off-the- results to calibrate the tests MAT 6 from the CMT but shelf" norm- Linked results using no good validity checks referenced equipercentile equating.
|
From page 28... ...
to allow for accurate procedures to compare data function for males was used comparisons between from state tests and NAEP rather than the equating state academic The standardized test results function for females performance and the were converted to the NAEP Linking standardized tests to national performance scale using the 1990 data and NAEP using equipercentile levels measured by resulting conversion tables equating procedures is not NAEP. were then applied to the 1992 sufficiently trustworthy to use for data other than rough approximations Examined content match between Designing tests in accordance standardized tests and NAEP with a common framework and re-analyzed data using might make linking more subsections of the standardized feasible tests and NAEP
|
From page 29... ...
Used two different types of linking for separate facets of the study. A socially moderated linkage was obtained by setting standards independently on the ITBS using the same achievement-level descriptions used to set the NAEP achievement levels.
|
From page 30... ...
district, in all four states. state assessments to Participating states used Students with the same state NAEP; and to different assessments in their assessment score, would be demonstrate that it state testing programs.
|
From page 31... ...
TIMSS. Validated the linking functions using data provided by states who participated in both state level NAEP and state level TIMSS but were not included in the development of the original linking function
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.