Testing Teacher Candidates The Role of Licensure Tests in Improving Teacher Quality (2001) / Chapter Skim
Currently Skimming:
Test 3: Middle School: English/Language Arts Test
Test 3: Middle School: English/Language Arts Test
Pages 393-414
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 393... ...
TEST AND ASSESSMENT DEVELOPMENT . Purpose: ETS describes the purpose of this test as to measure "whether an examinee has the knowledge and competencies necessary for a beginning teacher of English Language Arts at the middle school level" (Tests at a Glance: Middle School, p.
|
From page 394... ...
analyzing the results of the survey. The initial step revising the job analysis inventory used for secondary school teachers involved ETS test development staff, who made appropriate modifications in the inventory (e.g., made instructions relevant to middle grades, changed English to language arts)
|
From page 395... ...
: ETS has provided only a generic description of the test development procedures for all its licensure tests. In addition to the generic description, ETS has developed its own standards (The ETS Standards for Quality and Fairness, November 1999)
|
From page 396... ...
The committee produces draft test specifications and begins to articulate the form and structure of the test. Step 1A: Confirmation (Job Analysis)
|
From page 397... ...
(The validation study is described in the November 1992 ETS document Multistate Study of Aspects of the Validity and Fairness of Items Developed for the Praxis Series: Professional Assessments for red ~ TM Beginning earners . Comment: The procedures described by ETS in the Validation and Standard-Setting Procedures Usedfor Tests in the Praxis Series™ for examining the congruence between test items and the table of specifications and their relevance to practice are consistent with sound measurement practice.
|
From page 398... ...
The grade level that the panelists taught was not delineated for each test in the study; however, there were panelists in the overall study at the grade levels for which this test is designed (teachers in grades 5 to 9~. This lends some potential credibility for the congruence of the items to the specifications if the items reviewed were in the pool of items available for use on this test (but there is no assurance of that)
|
From page 399... ...
The process is described in detail in the materials provided. (Data describing the October 1998 test administration are contained in ETS, August 1999, Test Analysis Subject Assessment Middle School English Language Arts Form 3UPX1.)
|
From page 400... ...
: The Test Analysis Subject Assessment document does not contain data on test takers' speed. Included are frequency distribution, means, and standard deviations of traditional item difficulty values,2 observed and equated deltas,3 and biserial4 correlations.
|
From page 401... ...
and the item score that is hypothesized to have an underlying continuous distribution that is dichotomized as right or wrong. Biserial correlations are computed only when the percentage correct is between 5 and 95 and more than half the analysis sample reaches the item.
|
From page 402... ...
The ETS standards require that any cut score study be documented. The documentation should include information about the rater selection process, specifically how and why each panelist was selected, and how the raters were trained.
|
From page 403... ...
The recommended cut score is the average cut score for the entire group of panelists. For the constructed-response items, the same panelists who performed the modified Angoff method used one of two methods for setting a passing score on the constructed-response items.
|
From page 404... ...
It should be interpreted to mean that no reports from individual states describing this aspect of testing were contained in the materials provided. If ETS uses the procedures described for setting a recommended cut score in each state that uses this test, the process reflects what is considered by most experts in standard-setting to be sound measurement practice.
|
From page 405... ...
It should be interpreted to mean that no specific reports from user states about this aspect of the validity of the test scores were found in the materials provided. As noted earlier, the multistate validity study may provide limited information about the content validity of this test.
|
From page 406... ...
These data would be expected to be in the statistical report (ETS, February 1999, Test Analysis Middle School English Language Arts)
|
From page 407... ...
· Examinees have comparable questions/tasks (e.g., equating, scaling, calibration) : The ETS standards and other materials provided suggest that substantial efforts are made to ensure that items in this test are consistent with the test specifications derived from the job analysis.
|
From page 408... ...
The risk of contamination may be moderated somewhat by the use of multiple-choice items to provide measures of all dimensions within the test specifications and to the extensive item review process that all such tests are subject to if the ETS standard test development procedures are followed.
|
From page 409... ...
The large number of administration sites suggests that there will be some variability in the appropriateness of accommodations; however, it is clear that efforts are made (providing detailed manuals, announced and unannounced site visits by ETS staff) to ensure at least a minimum level of appropriateness.
|
From page 410... ...
The personnel requirements for test administration are also described in the manuals. Comment: The logistical, space, and personnel requirements are reasonable and consistent with what would be expected for any similar test.
|
From page 411... ...
ETS may also cancel a test score if it finds that a discrepancy in the process has occurred. The score reports to recipients other than the examinee are described as containing isnformation about the status of the examinee with respect to the passing score appropriate to that recipient only (e.g., if an examinee requests that scores be sent to three different states, each state will receive pass/fail status only for itself)
|
From page 412... ...
related to such things as descriptions of the program, the need for using justifiable procedures in setting passing scores, history of past litigation related to testing, and the need for validity for licensure tests. Comment: The materials available to applicants are limited but would be helpful in preparing applicants for taking this test.
|
From page 413... ...
TEST 3 413 extensive review procedures)
|
From page 414... ...
descnbe the EA/ELA assessment as follows: 2 The NBPTS assessments are currently being revised. The final publication copy of this report will include information on the revised NBPTS assessments and an updated example for the English Language Arts assessment..
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.