Redesigning the U.S. Naturalization Tests Interim Report (2004) / Chapter Skim
Currently Skimming:
3 The Test Development Process
3 The Test Development Process
Pages 12-23
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 12... ...
- Build test specifications. Create potential test items and scoring rubrics.
|
From page 13... ...
According to the Standards, once the content frameworks are developed, the test developer can assemble a set of potential test items that meets the test specifications. Usually test developers create a larger set of items than will eventually be needed, to allow some items to be discarded if it is found that they do not function as intended.
|
From page 14... ...
Pilot test items are not used to obtain scores for test takers, but only to provide data for test development purposes. Items that meet the test specifications, according to pilot test data, are assembled into a test, or into multiple alternative test forms.
|
From page 15... ...
It includes tests of writing, social studies, science, literature and the arts, and mathematics. As described m the GED technical manual, the test development process for the GED includes content frameworks and test specifications, as well as many levels of item and test review and two rounds of field testing, as summarized below: External item writers draft test questions, then a GED staff test specialist edits or rejects them.
|
From page 16... ...
Early m the test development process, CAL convened a 10-member technical working group to provide advice throughout the development process. To prepare for the first cycle of field testing, CAL staff developed initial item and test specifications that were approved by the technical working group.
|
From page 17... ...
The purpose of the Phase 2 Pilot seems to be a feasibility study to try out different test forms and new standardized test administration procedures. However, such a pilot requires that numerous earlier, major steps in the process have been completed—including content frameworks, test specifications, development of scoring rubrics, and pilot testing of items to collect data to build equated test forms.
|
From page 18... ...
whether a language instruction intervention, targeted toward populations at risk of performing poorly on the redesigned tests, can systematically increase performance on the language proficiency tests; (4) the extent to which the redesigned test of English proficiency correlates with other widely used measures of language proficiency; and (5)
|
From page 19... ...
Because Section 312 of the immigration and Nationality Act does not clearly define such constructs as "reading and writing simple words and phrases" or the "understanding of the fundamentals of history," interpreting the law's intent becomes a judgment that must be made by those involved in the redesign effort. With a highly visible testing program such as the naturalization tests, it is important to make the process of developing the content frameworks, which serve as the basis for so much of the rest of test development, as public and transparent as possible.
|
From page 20... ...
A more conventional approach for developing the content standards would be a sequential one, in which the K-12 standards and textbooks are the starting point, the expert panel is the main source of Information, and then reactions from stakeholder groups and the public are sought and Integrated into the content framework, as deemed appropriate by an oversight body. In addition to these issues about the weighting and integration of the different sources of evidence about history and government content, the committee has concerns about the process that was used to form the history and government panel.
|
From page 21... ...
In our judgment, a highly credible expert panel for English language content should be part of the structure for the redesigned tests. Recommendation 3: Work on developing the content frameworks (including publishing the history and government framework in the Federal Register)
|
From page 22... ...
These advisory bodies should recommend a plan for building on or modifying the existing content framework, bearing m mind testing standards related to developing content frameworks. In addition, the oversight committee and a new English language content panel should review the process currently being used to draft the content framework for the English reading, wliting, and speaking tests.
|
From page 23... ...
An important issue related to making pass/fail decisions about naturalization applicants is whether exceptions, accommodations, or different passung scores will be allowed for special populations. Under the current testing regime, federal regulations require that officers must consider an applicant's background when choosing and phrasing history and government test questions and evaluating responses (referred to as "due consideration.")
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.