Testing Teacher Candidates The Role of Licensure Tests in Improving Teacher Quality (2001) / Chapter Skim
Currently Skimming:
Test 5: Biology: Content Knowledge, Part 1 and 2 Tests
Test 5: Biology: Content Knowledge, Part 1 and 2 Tests
Pages 437-460
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 437... ...
Both tests cover content in an introductory college-level biology course. Comment: Stating the purpose of the tests publicly and having it available for potential test takers are appropriate and consistent with good measurement practice.
|
From page 438... ...
The staff reviewed the literature, including skills required by various states, and drew on the Test Development Staff's own teaching experience. These 128 knowledge statements were then reviewed by an External Review Panel of nine practicing professionals (four working teachers, four college faculty members, and one consultant from the National Science Foundation)
|
From page 439... ...
of the 189 knowledge statements were considered eligible for inclusion on the various biology tests because they had importance ratings of 2.5 or higher on the five-point scale. Means for each item were calculated for each of four respondent characteristic categories (sex, region of the country, race, years of teaching experience)
|
From page 440... ...
: ETS has provided only a generic description of the test development procedures for all of its licensure tests. In addition to the generic description, ETS has developed its own standards (The ETS Standards for Quality and Fairness, November 1999)
|
From page 441... ...
3~. Comment: The procedures ETS has described are consistent with sound measurement practice.
|
From page 442... ...
(The validation study is described in the November 1992 ETS document Multistate Study of Aspects of the Validity and Fairness of Items Developedfor the Praxis Series: Professional Assessments for Beginning Teachers™.) Comment: The procedures described by ETS for examining the congruence between test items and the table of specifications and their relevance to practice are consistent with sound measurement practice.
|
From page 443... ...
Part 1 are contained in ETS, December 1996, Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 1 Form 3PPX, and summary comparative data for the three different forms of this test are in a single-page document, Comparative Summary Statistics BIOK1M (0231)
|
From page 444... ...
(Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 1 Form 3PPX, and Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 2 Form 3PPX.) Other means of examining stability across forms are to examine the distributional characteristics of the forms.
|
From page 445... ...
: Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 1 Form 3PPX includes summary data on Form 3PPX, which was first administered in November 1993. These data include information on test takers' speed (99.4 percent of examinees reached item 75)
|
From page 446... ...
: Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 2 Form 3PPX includes .
|
From page 447... ...
No equated deltas were calculated as this is the base form of the test. The biserial correlations ranged from a low of .06 to a high of .65.
|
From page 448... ...
If the items identified as functioning differently are not examined critically to determine their appropriateness for continued use or if no other action is taken, the DIF analysis serves no purpose. Unfortunately the ETS standards document was incomplete, so it was not possible to ascertain what the specific policies are regarding use of the results of the DIF analysis.
|
From page 449... ...
Qualifications and demographic characteristics of personnel: No information was found regarding the internal ETS sensitivity review of the items in the pool for these tests. The Fairness Review Panel that served as part of the multistate validity study made decisions regarding the deletion, modification, or inclusion of items.
|
From page 450... ...
If ETS uses the procedure described for setting a recommended cut score in each of the states that use these tests, the process reflects what is considered by most experts in standard setting to be sound measurement practice. There is some controversy in the use of the Angoff method, but it remains the most often used method for setting cut scores for multiple-choice licensure examinations.
|
From page 451... ...
The absence of information on specific content validity studies should not be interpreted to mean there are no such studies. It should be interpreted to mean that no specific reports from user states about this aspect of the validity of the test scores were found in the materials provided.
|
From page 452... ...
For both of these tests the base form is Form 3PPX, first administered in November 1993. (Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 1 Form 3PPX, and Test Analysis Subject Assessments and NTE Programs Specialty Area Tests Biology: Content Knowledge, Part 2 Form 3PPX.)
|
From page 453... ...
. Test security: Procedures for test security at administration sites are provided in the 1999-2000 Supervisor's Manual and the 1999-2000 Supervisor's Manual for Nonstandard Test Administrations .5 These manuals indicate the need for test security and describe how the security procedures should be undertaken.
|
From page 454... ...
Comment: Scores on this test are unlikely to be contaminated by examinees employing knowledge or skills other than those intended by the item writers. This is due largely to the multiple-choice structure of the items and to the extensive item review process that all such tests are subjected to if the ETS standard test development procedures are followed.
|
From page 455... ...
It only means that the information for this element of the evaluation framework was not found in the materials provided. Because ETS is the owner of the tests and is responsible for scoring and reporting the test results, it is clear it has some responsibility for handling an appeal from an examinee that results from a candidate not passing the test.
|
From page 456... ...
However, for Part 2, the advanced test, the fact that only 93 percent of examiners finish would seem to justify some modification in either the items, the testing time, or the passing scores. The fee structure is posted and detailed.
|
From page 457... ...
is appropriate. · Exposure to legal challenge: No information on this element of the evaluation framework was found in the materials provided.
|
From page 458... ...
The procedures reportedly used for test development, standard setting, and validation are all consistent with sound measurement practices. The fairness reviews and technical strategies used are also consistent with sound
|
From page 459... ...
Because it appears that both tests are changing in their psychometric characteristics (i.e., the basic test, Part 1, is becoming more difficult and the advanced test, Part 2, easier) , the equating procedure is a critical aspect of the testing process.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.