Reporting Test Results for Students with Disabilities and English-Language Learners Summary of a Workshop (2002) / Chapter Skim
Currently Skimming:
6. Effects of Accommodations on Test Performance
6. Effects of Accommodations on Test Performance
Pages 38-69
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 38... ...
to document the effects of the inclusion and accommodation policies on participation rates for students with special needs; (2) to examine the effects of accommodations on performance results; (3)
|
From page 39... ...
Comparisons of the S 1 and S3 samples allowed examination of the net effect of making both types of changes. Participation Rates for the 1996 Mathematics anal Science Assessments Mazzeo highlighted some of the major findings on participation rates based on the 1996 mathematics and science assessments of fourth and eighth graders.
|
From page 40... ...
DIF analyses involved comparing performances of accommodated and standard test takers on each administered item, looking for differences in the way the groups performed on the item when ability level is held constant.2 Comparisons of IRT model fit were handled by examining item fit indices estimated in the S2 sample (no accommodations) and the S3 sample (accommodations allowed)
|
From page 41... ...
They compared average scale scores for the entire sample, various demographic groups, students with disabilities, and English-language learners as calculated in each of the samples. Comparisons of group mean scale scores as calculated in the S2 and S3 samples revealed some differences.
|
From page 42... ...
The researchers devised a way to examine the differences in average scale scores in combination with the change in participation rates from S2 to S3. For the fourth grade sample, this analysis revealed a negative relationship between average scores and participation rates.
|
From page 43... ...
Analyses were again conducted to examine differences in average scale scores in combination with changes in participation rates. As with the 1998 assessment, a negative relationship between average scale scores and participation rates was found when results were compared for S2 and S3 for eighth graders.
|
From page 44... ...
Figure 6-1 portrays a visual depiction of the 2 x 2 experimental design used to test for this interaction effect. An interaction effect would be said to exist if the mean score for examinees in group C were higher than the mean score for group A, and the mean scores for groups B and D were similar.
|
From page 45... ...
Effect sizes were calculated by comparing students' mean performance under accommodated and nonaccommodated conditions. Comparisons of the means for the two groups under the two accommodation conditions revealed a large effect (.88)
|
From page 46... ...
Effects of Accommodations on Mathematics Test Scores The second study Elliott discussed was an experimental analysis of the effects of accommodations on performance on mathematics items (Schulte, Elliott, and Kratochwill, 20001. The participants included 86 fourth graders, half of whom had disabilities.
|
From page 47... ...
In addition, the survey results indicated that, under the extended time conditions, the majority of students felt more comfortable and less frustrated, were more motivated, thought they performed better, thought the test seemed easier, and preferred taking the test with that accommodation. Effects of Oral Administration on Reading Test Scores The final study Elliott described examined the effects of using readaloud accommodations on a Terra-Nova reading test administered to eighth graders (McKevitt and Elliott, 20011.
|
From page 48... ...
Effect sizes were calculated by subtracting each student's nonaccommodated test score from his or her accommodated test score, then dividing by the standard deviation of the nonaccommodated test scores for the entire sample. The average effect sizes associated with the use of teacher-recommended accommodations were small for students with disabilities (.25)
|
From page 49... ...
The results showed that the responding teachers correctly identified allowable accommodations about half the time, and special education teachers responded correctly about as often as general education teachers. Another survey examined the extent to which teachers identified appropriate accommodations for students.
|
From page 50... ...
However, scores for special education students who received the read-aloud accommodation were statistically significantly higher than the scores for those who did not receive this accommodation. The researchers concluded that these findings confirmed the presence of a significant interaction effect and suggested that the read-aloud accommodation is valid for mathematics items.
|
From page 51... ...
Effects of Language Simplification on Mathematics Test Scores In another study, Tindal and his colleagues examined the effects of providing a simplified-language version of a mathematics test to students with learning disabilities (Tindal, Anderson, Helwig, Miller, and Glasgow, 19991. Study participants were 48 seventh graders two groups of general education students enrolled in lower mathematics classes (16 per group)
|
From page 52... ...
Tindal and his colleagues have also conducted factor analyses4 to study the factor structure of word-processed and handwritten response formats (Helwig, Stieber, Tindal, Hollenbeck, Heath, and Almond, 19991. For this study, 117 eighth graders (10 of whom were special education students)
|
From page 53... ...
Results from the earlier study indicated that Kentucky had been successful with including most students with disabilities, but several issues were identified for further investigation. In particular, the 1995 study revealed questionably high rates of providing accommodations, implausibly high mean scores for groups of students with disabilities, and some indication of differential item functioning in mathematics.
|
From page 54... ...
In 1995, fourth graders with disabilities performed nearly as well as those without disabilities, with differences ranging from .1 of a standard deviation lower in science to .4 of a standard deviation lower in mathematics. In 1997, means for fourth graders with disabilities ranged from .4 of a standard deviation lower than those without disabilities on the open-ended science item to .7 of a standard deviation lower on the science multiple-choice items.
|
From page 55... ...
The second finding of note related to differences in the way students with disabilities who received accommodations and those who did not performed on certain item types. Preliminary analyses revealed that elementary students with disabilities tended to score lower on multiple-choice items than on open-response items in all subjects except reading, and the lower scores appeared to be attributable to students with disabilities who received accommodations.
|
From page 56... ...
Their analyses revealed that the higher levels of performance were associated with learning disabled students' use of dictation, oral presentation, and paraphrasing accommodations. In 1995, fourth graders with learning disabilities who received dictation in combination with oral presentation and/or paraphrasing outperformed students without disabilities on the reading, science, and social studies open-response items.
|
From page 57... ...
These analyses involved two comparisonsone between students with disabilities who did not receive accommodations and students without disabilities; the other between students with disabilities who received accommodations and students without disabilities. The analyses revealed some evidence of DIF on the mathematics tests for students with disabilities who received accommodations (but not for those who were not accommodated)
|
From page 58... ...
Her analyses raised some interesting questions about verbal load and other kinds of reasoning processes. She speculated that cognitive analyses of test items with both non-disabled students and students
|
From page 59... ...
Studies on Linguistic Modification Abedi first described several studies that examined the effects of using linguistic modification as an accommodation. In one study, 946 eighth graders (53 percent English-language learners and 47 percent native or "fluent" English speakers)
|
From page 60... ...
All participants responded to a background questionnaire and took a NAEP reading test. The highest group mean scores were observed under the glossary plus extended time accommodation.
|
From page 61... ...
This finding was true for both English-language learners and native English speakers. Among the linguistic features that appeared to cause problems for students were low-frequency vocabulary and .
|
From page 62... ...
Further investigation into these findings suggested that students tended to perform best on mathematics tests that were in the same language as their mathematics instruction. That is, the Hispanic English-language learners who received their mathematics instruction in English or sheltered English scored higher on the English version of items (standard or linguistically modified)
|
From page 63... ...
English-language learners scored highest when they used the customized dictionary, and there was no impact on the performance of native English speakers. Abedi interpreted the findings as suggesting that the use of the customized dictionary was effective and did not alter the construct being measured.
|
From page 64... ...
Because reducing the level of linguistic complexity of test questions helps to narrow the performance gap between English-language learners and native English speakers, he believes this should be a priority in the development of all large-scale assessment programs. Abedi finds that the research demonstrates that student background variables, including language background, are strong predictors of performance.
|
From page 65... ...
EFFECTS OF ACCOMMODATIONS ON TEST PERFORMANCE SUMMARY OF RESEARCH FINDINGS 65 To help the reader assimilate the information presented in this chapter, Tables 6-1 through 6-3 highlight the key features and findings from the studies discussed in detail by the third panel of workshop speakers. Tables 6-1 and 6-2 highlight findings for students with disabilities and Englishlanguage learners, respectively.
|
From page 66... ...
66 Cal to to o o C~ so o o o ._ Cal o ._ Cal ._ ._ ._ ._ be .
|
From page 67... ...
67 ~ ~ , , ,' it , ~ C ' ~ by .
|
From page 68... ...
68 Cal ._ to o .
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.