Grading the Nation's Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress (1999)

Accommodations/adaptations, 51, 88-89, 91-112 (passim), 191

Americans with Disabilities Act, 88

Goals 2000: Educate America Act, 10, 88, 96

individualized education plans, 88, 94, 99-100

Individuals with Disabilities Education Act, 88

items read orally to poor readers, 13, 95, 99, 105

test administration procedures, 91, 92, 99-100

Accountability, 2, 10, 19, 23, 27, students with disabilities/English-language learners, 88, 91, 94, 107, 109, 110-111, 112

Achievement-level descriptions, 114, 128, 134-139, 141, 152-154, 163-173 (passim)

preliminary, 119, 134-135, 165, 170, 176-179, 182, 183, 256, 257, 259, 260

Achievement levels, 2, 7, 9, 10, 17, 20, 25, 72, 73, 162-184, cost factors, 8

definitional issues, 54, 124, 137-138, 139, 169-170, 186

frameworks and assessment development process, 115, 116, 117, 121 , 124-125, 128, 134, 135-140, 143-147 (passim), 152-154, 158, 159, 160, 192, 194, 195, 196

reporting, 2, 6, 7, 25-31, 32, 33, 72, 73, 135, 162, 163-165

special studies, 16, 57, 110

use of NAEP results, 2, 31, 171, 176, 180, 182, 184, 246

see also Mathematics achievement;

Performance standards;

Reading achievement;

Science achievement;

Writing achievement

Achievement-level setting, 2, 7, 31, 48, 116, 134-135, 152, 162, 163-168, 171, 176, 179-184, 186, 246, 256-261

anchoring, 2, 170-171, 173, 177, 257-261

Angoff procedure, 165, 166, 168

NCES role, 167, 175-176, 187, 196

see also Cutscores

Administration procedures, testing, 12-13, 65

costs, 78

grade-vs age-level testing, 13, 14

individual testing, 12-13, 99

item response time, 13, 62, 99, 128, 130-131

items read orally due to poor readers, 13, 95, 99, 105

professional administrators, 13, 82, 86

state vs national NAEP, 82, 85

students with disabilities/English-language learners, 91, 92, 99-100

see also Group testing

Administration schedules, testing, 18, 121, 191, 192-193, 195

Administrative factors, see Governance factors;

School-level measures

Advanced achievement levels, 2, 7, 64, 147, 152, 163, 164-171, 173 , 174, 175, 180, 183, 186, 256, 259, 260

Advanced placement, 165, 167, 169, 174

Advisory Council on Education Statistics, 53

African Americans, 44, 62

Age factors, 12, 13, 79

grade-vs age-level testing, 13, 14, 192-193

see also Grade-level factors

Alexander-James Panel, 2, 15

American College Testing, 19, 168-169, 190

American Institutes for Research, 190

Americans with Disabilities Act, 88

Analysis methods, 20, 189, 193, 196

coordinated system of education indicators, 45, 49-51 (passim), 55-56

costs, 78

current NAEP, 35, 58, 62, 63-64, 65, 67, 68-70, 71, 78

frameworks and assessment development process, 116, 117

reporting metrics, 72

secondary data, 42

streamlining NAEP design, 4, 14, 62, 63-64, 65, 67, 68-70, 71, 78 , 79

students with disabilities/English-language learners, 101, 104, 108, 110

text analysis, 43, 49, 219-220

video analyses, 43, 46, 48, 194

see also Computer-based testing and analysis;

Content coverage and analysis;

Item analysis and development

Anchoring, 2, 170-171, 173, 177, 257-261

Angoff procedure, 165, 166, 168

Applied Measurement in Education, 11

Arts achievement, 193-194

Assessment development process, 2, 4, 5-6, 33, 67, 92, 114-161, 190-191, 194, 195, 220-236

achievement levels, 115, 116, 117, 121, 124-125, 128, 134, 135-140 , 145-147, 152-154, 158, 159, 160, 192, 194, 195, 196;

see also ''mathematics achievement'' and "science achievement" infra

constructed-response items, 115, 122, 123, 128-129, 132, 146, 149 , 150, 153-154

contractors, 115, 116, 118, 119, 136, 148-150, 190, 192-193, 195

defined, 115

funding, 149-150, 151, 155, 191

governance, 117-118, 134-135

interpretive information, 114, 116, 137, 153, 154-157, 159, 160-161

item analysis and development, 117, 118-119, 122-123, 125, 128-137 , 142, 149-155, 160, 219-236, 256-257

knowledge and skills measures, 115, 121, 125, 126, 127, 128, 131-133, 138, 139, 140, 156, 159, 233, 256-257

large-scale assessments, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194

main NAEP, 114, 118-121, 126-128, 136, 194, 195, 259

mathematics achievement, 118, 121, 122-124, 126-127, 132, 136, 142

NCES role, 115, 116, 118, 122, 134, 148-149, 154

performance standards, 114, 115, 118, 123, 125, 133, 142-143, 146 , 149, 155, 159, 220, 257, 259-260

reading achievement, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230

reporting procedures, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221

science achievement, 116, 118, 126, 129-130, 132-133, 134, 136, 30, 132-133, 134, 136, 142, 143-144, 231-236

scoring, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149 , 150, 153-154, 156, 160, 161, 219-220, 226-227, 234-235, 257

summary scores, 116, 117, 121, 128, 139, 152, 154

task measures, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154 , 160, 219-220

see also Field tests;

Item analysis and development

Assessment in Transition: Monitoring the Nation's Educational Progress, 25, 124

Association of State Assessment Programs, 97

Background variables, 34

instructional, 65

students, 35, 40, 42, 64, 95, 119, 122, 149, 188, 190, 220

teachers, 40, 41, 43

Balanced incomplete blocks, 62, 63, 66

Barometer information, see Descriptive information

Bias, see Error of measurement

Bilingual education, 91, 98-101 (passim), 104, 105

translated versions of NAEP, 95, 97, 100, 101, 111

Black persons, see African Americans

Center for Equity and Excellence, 98

Civics, 18, 61, 136

Class size, 26, 44

Cognitive factors and theory, 6, 7, 19-20, 128, 133, 137-141, 143-146, 166

see also Knowledge and skills measures;

Learning theory;

Problem solving

Common Core of Data, 36-37, 51

Computer-based testing and analysis, 32, 63, 148, 194

The Condition of Education, 35

Constructed-response items, 80, 136, 166, 174

frameworks and assessment development process, 115, 122, 123, 128-129, 132, 146, 149, 150, 153-154

main vs trend NAEP, 79

national NAEP, 80

nonresponse rates, 123

Content coverage and analysis, 12, 27, 48-49, 70, 79, 118

text analysis, 43, 49, 219-220

Contextual data, see Interpretive information

Contracts and contractors, 17, 31, 68, 82, 116, 118, 136, 150, 166 , 168, 177, 191, 195, 196

administration of testing, state vs national, 82

Educational Testing Service, 14, 17, 31, 93, 118, 119, 148-149, 150, 190

frameworks and assessment development process, 115, 116, 118, 119 , 136, 148-150, 190, 192-193, 195

KPMG Peat Marwick LLP, 25, 66, 69, 70, 78

NAEP, redesign/transitional, 148, 175, 187, 189-196 (passim)

National Computer Systems, 119

RAND, 40, 46

scheduled contracts and administration by framework, 192-193, 195

WESTAT, 17, 119

Coordinated system of education indicators, 4, 6, 8, 20, 22-55, 73, 158, 186, 187, 188

analysis methods, 45, 49-51 (passim), 55-56

costs of, 51-52, 53

curriculum, 23, 40, 41, 43, 54

data collection, 4, 6, 8, 20, 22-55, 106

Department of Education role, 23-24, 34, 50, 53-54

frameworks, 24, 33, 38, 40, 47, 51-55 (passim)

funding, 23, 41, 42, 44, 53

governance, 45, 51-54

NAGB, 22-25, 31, 187

international components, 42, 43, 46, 158, 188

interpretive information, 4, 23, 24, 25, 27-55, 45, 115

knowledge and skills measures, 23-24, 47

large-scale assessments, 24, 34, 47, 51, 54, 188

NCES, 31, 34-38, 40-41, 45-46, 49-54, 187

new paradigm NAEP, 46-47, 48, 55, 84, 85, 109, 110, 146-147, 151, 158, 187-197

pedagogy, 23, 40-43 (passim), 46, 47-49, 54

reporting procedures, 25-26, 73

sampling, 45, 46, 47, 49, 50-51, 107

skills measures, 24, 47

standards, general, 23, 45

state-level factors, 45-46

students with disabilities/English-language learners, 106, 107

task measures, 23, 24, 33, 47, 54

validity issues, 40, 46

Core NAEP, 7, 47, 57, 78, 80, 82, 84, 85, 147-149, 153, 155, 158, 181-182, 187, 188, 194

students with disabilities/English-language learners, 109, 110, 112

see also Mathematics achievement;

Reading achievement;

Science achievement;

Writing achievement

Cost and cost-benefit factors, 8, 10, 78

elementary/secondary school expenditures, data on, 35

indicator systems, 38, 51-52, 53

NAEP design, general, 3, 8, 62, 78, 84, 149-150, 151

national/state systems separate, 66, 78, 82, 85

sampling designs, 62, 66

state-level indicators, 62, 66, 78, 82, 85

students with disabilities/English-language learners, 8, 107-108, 111

trend NAEP, 78

see also Accountability;

Funding

Council of Chief State School Officers, 17, 40, 46, 53, 96, 98

Council of the Great City Schools, 53

CSEI, see Coordinated system of education indicators

Cultural bias, 63

Curriculum, 6

cognitive theory and, 19-20

coordinated system of education indicators, 23, 40, 41, 43, 54

high school, 80-81

NAEP design and, 12, 16, 20, 58, 79, 80-81

special studies, 16, 57

standards, 10, 179, 153

students with disabilities/English-language learners, 111

Curriculum and Evaluation Standards for School Mathematics, 10, 122

Cutscores, 64, 166, 167-171, 173-177 (passim), 180, 182, 183, 256 , 260

Databases, 50-52, 196

Data collection, general, 4, 11, 14, 20, 23, 196, 259

coordinated system of education indicators, 4, 6, 8, 20, 22-55, 106

frameworks, 115, 150

mixed methods of, 43, 45, 47-49, 54

NAGB objectives, 25

national vs state, 65

streamlining NAEP design, 4, 7, 65, 68, 73, 76, 79-82, 84, 190, 196

students with disabilities/English-language learners, 87, 96, 104 , 106, 107, 108, 112, 113

see also Large-scale assessments;

Sampling

Data presentation, see Reporting procedures

Demographic factors, 1, 152, 19

performance measures, 27, 163

reporting, 1-2, 27, 31, 34, 163

socioeconomic status, 14, 50

poverty, 14, 44

social indicator systems, 3, 24, 38-39, 152, 153

summary scores, 14

students with disabilities/English-language learners, 89-90

see also Age factors;

Gender factors;

Race/ethnicity;

Urban/rural

factors

Department of Education, 1, 3, 187

coordinated system of education indicators, 23-24, 34, 50, 53-54

English-language learners, 105, 106

students with disabilities, 106, 109

see also Contracts and contractors;

National Center for Education

Statistics;

terms beginning "Office of…"

Department of Education Organization Act, 88

Descriptive information, 3, 8, 27-32, 33, 57, 72, 114, 128, 164, 197

cost factors, 8

students with disabilities, 101

Design/Feasibility Team, 66, 72, 78-79, 190

Difficulty level, see Item difficulty

Dimensionality of data, 63, 70

Disabilities, see Learning disabilities;

Students with disabilities

Dropouts, 57, 81, 86

Early Childhood Longitudinal Study, 36-37, 46, 110, 188

Economic factors, see Cost and cost-benefit factors;

Funding;

Poverty;

Socioeconomic status

Educating One and All: Students with Disabilities and Standards-Based Reform, 91, 94

Educational indicators, 4, 9, 10, 19, 154

summary scores, 128

see also Coordinated system of educational indicators

Educational Longitudinal Study, 36-37

Educational Testing Service, 14, 17, 31, 93, 118, 119, 148-149, 150, 190

Education Commission of the States, 14

Education Counts, 38

Education reform, 2, 9-10, 19, 20, 25-27 (passim), 118, 153

coordinated system of education indicators, 41

see also Accountability

Education Week, 172

Elementary and Secondary Education Act, 14

English-language learners, 7-8, 14, 19, 20, 26, 87-113, 134, 186, 191

accommodations/adaptations, 51, 88-89, 91-112 (passim), 191

analysis methods, 101, 104, 108, 110

bilingual education, 91, 98-101 (passim), 104, 105

translated versions of NAEP, 95, 97, 100, 101, 111

coordinated system of education indicators, 106, 107

core NAEP, 109, 110, 112

cost factors, 8, 107-108, 111

data collection, 87, 96, 104, 106, 107, 108, 112, 113

defined, 87(n. 1)

demography of, 89-90

educational indicators, 88, 106, 114

field tests, 98-101

funding, 108, 109

Goals 2000: Educate America Act, 10, 88, 96

grade-level factors, 90

historical perspectives, 5, 92-98

identification/classification, 102, 104-105, 106, 113

inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191

interpretive information, 5, 87, 99, 106, 112

large-scale assessments, 88, 89, 91-93, 96, 97, 98, 104, 105, 106-109, 110, 112-113

legislation, 10, 88, 96, 106, 108

mathematics achievement, 98-101, 103, 110

NCES role, 91, 93, 94, 97, 105, 106, 109, 191-192

numbers by grade level, 90

participation, 5, 19, 57

performance standards, 87, 89, 94

reading achievement, 94, 110

regional factors, 89-90

reporting procedures, 108, 111

sampling, 101, 102, 103, 106, 107

school-level measures, 89-90

skills measures, 104-105

standards, general, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113

state role, 88, 93, 94, 97, 98, 104, 106, 109, 110

summary scores, 106

test administration procedures, 91, 92, 99-100

TIMSS, 44

translated versions of NAEP, 95, 97, 100, 101, 111

urban/rural factors, 89-90

validity issues, 88, 91, 95, 106, 109-111

see also Hispanic students;

Participation, students with disabilities/English-language

learners

Equality of Opportunity Surveys, 50

Error of measurement, 63, 69, 82-83, 86, 118, 175

probability sampling, 62, 64, 66, 68-69, 175

Estimation procedures, 58, 76, 80, 86, 175

Evaluative information, 3, 4, 7, 8, 11, 27-33 (passim), 57

teacher evaluation of NAEP, 77-78, 129-130

see also Achievement levels;

Performance standards

Extended-response items, 128, 132, 136, 140, 149, 150, 155, 190-191 , 194, 220, 225, 226, 229, 230

Family factors, 27, 34, 41, 42, 44

Federal government

national curriculum, 13-14

standards, general, 10

see also Department of Education;

Funding;

Legislation

Field tests, 51, 117, 118, 119, 133-134, 136, 150, 194

reporting procedures, tables/displays, 71

Fit of items, 63, 69, 70, 101, 118, 122-123, 125, 128-132, 220, 221, 229

Focused Reports, 65

Focus on NAEP, 65

Foreign countries, see International perspectives;

specific countries

Foreign-language speakers, see English-language learners

Frameworks, 2, 4, 5-6, 17, 114-161, 165, 194, 257

achievement levels, 115, 116, 117, 121, 124-125, 128, 134, 135-140 , 145-147, 152-154, 158, 159, 160, 192, 194, 195, 196;

see also other "mathematics achievement" and "science achievement" infra

analysis methods, 116, 117

constructed-response items, 115, 122, 123, 128-129, 132, 146, 149 , 150, 153-154

contractors, 115, 116, 118, 119, 136, 148-150, 190, 192-193, 195

coordinated system of education indicators, 24, 33, 38, 40, 47, 51-55 (passim)

data collection, 115, 150

design streamlining, 57, 58-61, 62, 65-66, 77

funding, 149-150, 151, 155, 191

governance, 117-118, 134-135 NAGB, 116, 118, 127, 194, 196

interpretive information, 114, 116, 137, 153, 154-157, 159, 160-161

item analysis and development, 117, 118-119, 122-123, 125, 128-137 , 142, 149-155, 160, 219-236, 256-257

knowledge and skills measures, 115, 121, 125, 126, 127, 128, 131-133, 138, 139, 140, 156, 159, 233, 256-257

large-scale assessments, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194

main NAEP, 114, 118-121, 126-128, 136, 194, 195, 259

mathematics achievement, 118, 121, 122-124, 126-127, 132, 136, 142

national NAEP, 59-60

NCES role, 115, 116, 118, 122, 134, 148-149, 154

performance standards, 114, 115, 118, 123, 125, 133, 142-143, 146 , 149, 155, 159, 220, 257, 259-260

reading achievement, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230

reporting procedures, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221

scheduled contracts and administration by framework, 192-193, 195

science achievement, 116, 118, 126, 129-130, 132-133, 134, 136, 142, 143-144, 231-236

scoring, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149 ,

summary scores, 116, 117, 121, 128, 139, 152, 154

task measures, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154 ,

trend NAEP, 77-78, 192, 194

From Data to Information, 40, 41, 49-50

Funding, 19

coordinated system of education indicators, 23, 41, 42, 44, 53

framework and assessment development process, 149-150, 151, 155, 191

students with disabilities/English-language learners, 108, 109

see also Accountability;

Contracts and contractors;

Cost and cost-benefit factors

Gender factors, 2, 68, 152, 169

Geography achievement, 18, 61, 136, 166

Goals 2000: Educate America Act, 10, 88, 96

Governance factors, 17-19

coordinated system of education indicators, 45, 51-54

frameworks and assessment development process, 117-118, 134-135

reporting, 19, 31, 72, 164, 172, 190

see also National Assessment Governing Board

Government role, see Federal government;

Local government;

State government

Grade-level factors, 13, 14, 15, 31, 172-173, 192-193

see also Age factors

Graduate Record Exam, 93

Group testing, 12-13

group tasks, 47, 138

Hawkins-Stafford Elementary and Secondary School Improvement Amendment , 35

High School and Beyond, 31, 46

High school students, 44, 80-81, 85, 86, 186

dropouts, 57, 81, 86

Hispanic students, 44, 62, 95, 99

bilingual education, 91, 98-101 (passim), 105, 110

translated versions of NAEP, 95, 97, 100, 101, 111

Historical perspectives, 1-2, 9-15, 19, 163

achievement measures, 4

contextual data, 24

indicator systems, 38

sociopolitical context, 9-11

students with disabilities/English language learners, 5, 92-98

History, as academic subject, 18, 32, 52, 61, 136, 166

Improving America's Schools Act, 2, 10, 11, 17, 24, 88, 122, 162, 175, 196

Improving Schooling for Language-Minority Children: A Research Agenda , 91

Inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191

The Inclusion of Students with Disabilities and Limited Proficient Students in Large-Scale Assessments: A Summary of Recent Progress , 92

Indicator systems

costs of, 38, 51-52, 53

general, 38-39

purposes saved by 1, 3-4, 9, 10, 27-32, 34, 35-39, 42-45, 186;

see also Accountability

social, 3, 24, 38-39, 152, 153

see also Coordinated system of educational indicators

Individualized education plans, 88, 94, 99-100

Individual-level measures, see Student-level measures

Individuals with Disabilities Education Act, 88

Individual testing, 12-13

students with disabilities/English language learners, 99

Institute on Educational Governance, Finance, Policy Making, and Management, 53

Institute on Student Achievement, Curriculum, and Assessment, 52

Instruction, see Pedagogy

Instructional Reports, 65

Integrating Federal Statistics on Children, 40

International perspectives, 2, 15, 85, 110

coordinated system of education indicators, 42, 43, 46, 158, 188

sociopolitical trends, 9-10

TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259

Interpretive information, 6, 9, 19

cost factors, 8

coordinated system of education indicators, 4, 23, 24, 25, 27-55, 45, 115

frameworks and assessment development processes, 114, 116, 137, 153, 154-157, 159, 160-161

mass media, 9-10, 12, 25-30, 68, 71, 171

streamlining NAEP design, 4, 5, 57, 64, 65, 67-68, 71, 72, 73, 81 , 86, 190

students with disabilities/English-language learners, 5, 87, 99, 106, 112

see also Descriptive information;

Evaluative information

IRT, see Item response theory

Item analysis and development, 4, 7, 14, 62-64, 69, 178-179, 192

anchoring, 2, 170-171, 173, 177, 257-261

fit of items, 63, 69, 70, 101, 118, 122-123, 125, 128-132, 220, 221, 229

frameworks and assessment development process, 117, 118-119, 122-123, 125, 128-137, 142, 149-155, 160, 219-236, 256-257

matrix sampling, 13, 62, 68

balanced incomplete blocks, 62, 63, 66

see also Field tests

Item authoring, 256-257

Item difficulty, 13, 14, 100-101, 110, 119, 139, 166, 174-175, 179 , 238, 243, 245, 257

Item families, 146, 156-157, 160, 193, 237-255

Item mapping, 51, 169, 233, 257-259, 260-261

Item response theory, 62-64, 69, 72, 95, 101

Jackknife methods, 64, 69

Knowledge and skills measures, 5, 6, 12, 13, 19-20, 27, 77

coordinated system of education indicators, 23-24, 47

English-language learners, 104-105

frameworks and assessment development process, 115, 121, 125, 126 , 127, 128, 131-133, 138, 139, 140, 159, 233, 256-257

main NAEP, 4-5, 15-16, 77, 78-80

main vs trend NAEP, 77

see also Task measures

KPMG Peat Marwick LLP, 25, 66, 69, 70, 78

Language factors, see Bilingual education;

English-language learners

Large-scale assessments, 3-10 (passim), 21

achievement-level setting, 182

coordinated system of education indicators, 24, 34, 47, 51, 54, 188

frameworks and assessment development processes, 114, 125, 128, 133, 136, 147-149, 156, 158, 159, 161, 194

streamlining NAEP design, 56, 57, 70, 80, 84

students with disabilities/English-language learners, 88, 89, 91-93, 96, 97, 98, 104, 105, 106-109, 110, 112-113

Learning disabilities, 88, 95, 104, 108, 110

Office of Special Education and Rehabilitative Services, 91

Learning theory, 5-6, 19-20, 128, 137, 138-141, 151, 160

LEAs, see School districts

Legislation, 10

Americans with Disabilities Act, 88

Department of Education Organization Act, 88

Elementary and Secondary Education Act, 14

Goals 2000: Educate America Act, 10, 88, 96

Hawkins-Stafford Elementary and Secondary School Improvement Amendment , 35

Improving America's Schools Act, 2, 10, 11, 17, 24, 88, 122, 162, 175, 196

Individuals with Disabilities Education Act, 88

National Education Statistics Act, 52

Perkins Act, 88

students with disabilities/English-language learners, 10, 88, 96, 106, 108

Literacy, see English-language learners;

Reading achievement;

Writing achievement

Local Education Authorities, see School districts

Local factors and trends, 4

age-vs grade-level testing, 14

streamlining testing, 81-82

see also School-level measures

Local government

national curriculum, 13

standards, 56

Longitudinal studies, 31, 34, 36-37, 39, 42, 43, 46, 85, 110, 158

see also specific studies

Main NAEP

frameworks and assessment development process, 114, 118-121, 126-128, 136, 194, 195, 259

knowledge measures, 4-5, 15-16, 77, 78-80

mathematics achievement, 15, 17, 28-30, 65, 77-78, 80, 84, 118, 119, 125, 126-127, 132, 156, 157, 171, 192-193

national trends, 15-16, 56, 57, 58, 65

performance standards, 4-5, 15-16, 77, 78-80

reading achievement, 15, 17, 65, 80, 84, 118, 120, 125, 127-128, 133, 171, 192-193, 219

reporting, 163, 171

science achievement, 15, 17, 28-30, 65, 77-78, 80, 84, 116, 118, 121, 125, 126, 157, 168, 192-193, 259

state, 15, 16, 56, 57, 58, 65-66

students with disabilities/English-language learners, 103

trend vs, 4-5, 15-16, 58, 63, 64, 65, 73-80, 84, 85

writing achievement, 15, 17, 65, 80, 84, 192-193

Market-basket reporting, 72, 189

Mass media, 9-10, 12, 25-30, 68, 71, 171

Mathematics achievement, 7, 10, 27, 52, 58, 60, 65

English-language learners, 98-101, 103, 110

frameworks and assessment development process, 118, 121, 122-124, 126-127, 132, 136, 142

item families, 237-255

main NAEP, 15, 17, 28-30, 65, 77-78, 80, 84, 118, 119, 125, 126-127, 132, 156, 157, 171, 192-193

NAEP administration schedule, 18

National Council of Teachers of Mathematics, 122, 155, 193

national NAEP, 18, 60, 63, 73, 75, 77

new paradigm NAEP, 84

reporting, 28, 58, 59, 73, 171, 172

standards, 10, 166, 171, 172, 176

state NAEP, 82

students with disabilities, 103

TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259

trend NAEP, 15, 17, 18, 60, 63, 73, 75, 77, 80, 84, 102

trend vs main NAEP, 73, 75-80 (passim)

see also Achievement levels;

Core NAEP

Matrix sampling, 13, 62, 68

balanced incomplete blocks, 62, 63, 66

Minority groups, see Race/ethnicity

Mixed methods of data collection, 43, 45, 47-49, 54

Multiple assessment strategies, 4, 6, 7, 47, 57, 66, 73-80

Multiple-choice items, 12, 118, 128-129, 136, 146, 150, 174, 229, 230

guessing/nonresponse, 13, 64

item response theory, 63

main vs trend NAEP, 79

Multiple-methods NAEP, 6, 7, 8, 10, 46, 47, 66, 80, 81, 84, 85, 109, 110, 124, 147-156 (passim), 158, 181-182, 186, 188, 190, 193, 194

NAEP Facts, 65

NAEP Redesigned, 148-149

NAEP Report Cards, see Report Cards

National Academy of Education, 7, 25-26, 66, 67, 122, 137

performance standards, 167, 168, 174

students with disabilities/English-language learners, 92, 94, 96, 98

National Assessment Governing Board, 1, 2, 3, 11, 17, 65-66, 185, 189

achievement-level setting, 165-166, 168-171, 173-174, 175, 180, 182, 183, 187, 191, 196, 260

coordinated system of education indicators, 22-25, 31, 187

Design/Feasibility Team, 66, 72, 78-79, 190

frameworks and assessment development process, 116, 118, 127, 194, 196

NAEP objectives, general, 24-25, 78-79, 162, 189-190

reporting, 31, 72, 164, 172, 190

state government role, 25, 65-66

National Assessment of Educational Progress, see Core NAEP;

Main NAEP;

Multiple-methods NAEP;

National NAEP;

New paradigm NAEP;

State NAEP;

Trend NAEP

National Association of State Test Directors, 53

National Center for Education Statistics, 1-3, 17, 185, 189

achievement-level setting, 167, 175-176, 187, 196

coordinated system of education indicators, 31, 34-38, 40-41, 45-46, 49-54, 187

data collection, general, 36-37, 196

frameworks and assessment development process, 115, 116, 118, 122, 134, 148-149, 154

reporting procedures, 31, 154, 196

students with disabilities/English-language learners, 91, 93, 94, 97, 105, 106, 109, 191-192

see also Contracts and contractors

National Center for History in Schools, 52

National Center for Improving Student Learning and Achievement in Mathematics and Science, 52

National Center for Research on Evaluation, Standards, and Student Testing, 52, 97, 98

National Center for Research on the Organization and Restructuring of Schools, 53

National Center for the Improvement of Early Reading Achievement, 52

National Center for the Study of Teaching and Policy, 53

National Center on Education Outcomes, 92

National Center on Increasing the Effectiveness of State and Local Education Reform Efforts, 53

National Computer Systems, 119

National Council of Teachers of Mathematics, 122, 155, 193

National Education Longitudinal Study, 36-37, 44, 46, 50, 51, 110 , 188

National Education Statistics Act, 52

National Governors' Association, 15, 172

National Household Education Survey, 35-36

National Longitudinal Survey of Youth, 44

National NAEP, 15-16, 56, 57, 58-60, 63, 65, 73

administration of testing, 82

constructed-response items, 80

costs, 66, 78, 82, 85

frameworks, 59-60

main NAEP, 15-16, 56, 57, 58, 65

mathematics achievement, 18, 60, 63, 73, 75, 77

multiple assessments, 73-77

pedagogy, 16, 58

reading achievement, 16, 18, 59, 73-75, 77

reporting, 1-2, 14, 71

sampling, 59-60, 62, 66

science achievement, 18, 60, 63

state vs, 65, 81-84, 85

writing achievement, 16, 18, 59

National Research and Development Center on English Learning and Achievement, 52

National Science Education Standards, 10

National trends, 5, 19, 46-46

coordinated system of education indicators, 45-46

main NAEP, 15-16, 56, 57, 58, 65

NAEP results reporting, 1-2, 14

NAGB objectives, 24

sampling, 59-63, 68, 194

standards and goals, 10

streamlining, 81-84, 189

students with disabilities/English-language learners, 110

see also National NAEP

A Nation at Risk, 1, 9, 14

The Nation's Report Card, 15, 154, 172

NCES, see National Center for Education Statistics

New paradigm NAEP, 46-47, 48, 55, 84, 85, 109, 110, 146-147, 151, 158, 187, 189, 194-196

Newspapers, see Mass media

Non-native English speakers, see English-language learners

Nonresponse rates, 64, 80

Office of Bilingual Education and Minority Languages, 91

Office of Civil Rights, 91

Office of Education Research and Improvement, 46, 91, 94

Office of Special Education and Rehabilitative Services, 91

Office of the General Counsel, 91

Open-ended items, 22, 146

see also Extended-response items

Participation, students with disabilities/English-language learners , 5, 19, 57, 87-89, 91-113 (passim)

accommodations/adaptations, 51, 88-89, 91-112 (passim), 191

Goals 2000: Educate America Act, 10, 88, 96

Pedagogy

coordinated system of education indicators, 23, 40-43 (passim), 46, 47-49, 54

national NAEP, 16, 58

teacher background, 40, 41, 43

trend NAEP, 77

video techniques, instructional methods analysis, 43, 46, 48, 194

Performance standards, 3, 17, 19, 20, 23, 44, 48, 162-184, 196

advanced level, 2, 7, 64, 147, 152, 163, 164-171, 173, 174, 175, 180, 183, 186, 256, 259, 260

defined, 162-164

demographic factors, 27, 163

frameworks and assessment development process, 114, 115, 118, 123, 125, 133, 142-143, 146, 149, 155, 159, 220, 257, 259-260

history of, 15

main NAEP, 4-5, 15-16, 77, 78-80

reporting of, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165

students with disabilities/English-language learners, 87, 89, 94

see also Achievement levels;

Constructed-response items;

Cutscores;

Extended-response items;

Task measures

Perkins Act, 88

Policy Statement on Redesigning the National Assessment of Educational Progress, 11

Political factors, 9-11, 13-15, 19, 20, 23, 25-26, 31, 38-39

stakeholders, 3, 25, 80

see also Accountability;

Education reform

Poverty, 14, 44

Preliminary achievement-level descriptions, 119, 134-135, 165, 170, 176-179, 182, 183, 256, 257, 259, 260;

see also Achievement-level descriptions

Press, see Mass media

Private School Universe Study, 36-37

Problem solving, 47, 57, 121, 127, 136-137, 138, 140, 146

Quality Counts, 172

Race/ethnicity, 14, 50, 79, 169

African Americans, 44, 62

bilingual education, 91, 98-101 (passim), 105, 110

cultural bias, 63

Hispanics, 44, 62, 95, 99

translated versions of NAEP, 95, 97, 100, 101, 111

trend NAEP, 44, 79

RAND, 40, 46

Reading achievement, 7, 65, 166

frameworks and assessment development process, 118, 119, 120, 122, 123, 127-128, 132, 133, 136, 141-142, 219-230

items read orally due to poor readers, 13, 95, 99, 105

main NAEP, 15, 17, 65, 80, 84, 118, 120, 125, 127-128, 133, 171, 192-193, 219

NAEP administration schedule, 18

NAEP mandate, 24

national NAEP, 16, 18, 59, 73-75, 77

new paradigm NAEP, 84

oral, 16

reporting, 31-32

state-level trends, 26, 59

students with disabilities/English-language learners, 94, 110

text analysis, 43, 49, 219-220

time factors, reading passages, 122

trend NAEP, 15, 17, 18, 59, 63, 73-75, 77, 80, 84

see also Achievement levels;

Core NAEP

Reform of education, see Education reform

Reform Up Close, 40

Regional factors and trends

English-language learners, 89-90

NAEP results reporting, 1, 14, 19, 152

Report Cards, 8, 65, 71, 72, 73, 107, 121, 152, 154, 163, 170, 172

Reporting metrics, 72, 84, 86, 159, 162, 164, 171, 189

Reporting procedures, 1-2, 8, 20, 25-26, 64-65, 66, 67-68, 70-73, 86, 196

achievement levels, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165

coordinated system of education indicators, 25-26, 73

costs, 78

demographic factors, 1-2, 27, 31, 34, 163

field tests, tables/displays, 71

frameworks and assessment development process, 114, 117, 121, 128, 135, 145-146, 151, 154-160 (passim), 221

governance, 19, 31, 72, 164, 172, 190

market-basket reporting, 72, 189

mathematics achievement, 28, 58, 59, 73, 171, 172

national NAEP, 1-2, 14, 71

NCES role, 31, 154, 196

performance standards, 2, 6, 7, 25-31, 32, 33, 72, 73, 162, 163-165

reading achievement, 31-32

regional trends, 1, 14, 19, 152

scoring and scoring criteria, 72, see also Summary scores

standards-based, 2, 7, 25-26, 33, 162, 163-165, 167-168, 171-173, 182, 184

students with disabilities/English-language learners, 108, 111

state NAEP, 2, 10, 14, 15, 26, 31, 68, 71

timeliness, 67, 71, 152, 154

trend NAEP, 73, 163

see also Descriptive information;

Evaluative information;

Interpretive information

Rural areas, see Urban/rural factors

Sampling, 2, 4, 11, 13, 14, 17, 20, 45, 46, 47, 49, 50-51, 56, 57, 58-63, 64, 66, 68, 86, 161

coordinated system of education indicators, 45, 46, 47, 49, 50-51, 107

cost factors, 62, 66

dropouts, 81, 86

errors, 82-83

probability sampling, 62, 64, 66, 68-69, 175

jackknife methods, 64, 69

matrix sampling, 13, 62, 68

balanced incomplete blocks, 62, 63, 66

national NAEP, 59-60, 62, 66

national trends, other, 59-63, 68, 194

state-level, 59-62, 65-66, 68, 83, 85, 194

state vs national NAEP, 83, 85

students with disabilities/English-language learners, 101, 102, 103, 106, 107

time factors, 69

see also Demographic factors;

Error of measurement;

Race/ethnicity

Scaling, 4, 56, 66, 67, 72, 80, 101, 111, 117, 154, 163, 261

Scholastic Aptitude Test, 50, 93

School districts, 10, 82, 104

School-level measures, 27, 34, 39, 40, 41, 42, 45, 46, 47, 51

English-language learners, 89-90

Schools and Staffing Survey, 36-37, 41, 51

Science achievement, 7, 10, 27, 60, 65, 78, 80, 259

database, 50

frameworks and assessment development process, 116, 118, 126, 129-130, 132-133, 134, 136, 142, 143-144, 231-236

main NAEP, 15, 17, 28-30, 65, 77-78, 80, 84, 116, 118, 121, 125, 126, 157, 168, 192-193, 259

NAEP administration schedule, 18

national NAEP, 18, 60, 63

new paradigm NAEP, 84

standards, 10, 166, 168-171, 178-179

TIMSS, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259

trend NAEP, 15, 17, 18, 60, 63, 80, 84, 259

see also Achievement levels;

Core NAEP

Scoring and scoring criteria, 67, 192

frameworks and assessment development process, 114, 117, 118, 119, 125, 132, 133-134, 135, 136, 146, 149, 150, 153-154, 156, 160, 161, 219-220, 226-227, 234-235, 257

levels and types of knowledge, 6

multicategory, 67

nonresponse rates, 64, 80

reporting, 72

state vs national NAEP, 82-83

see also Cutscores;

Summary scores

Second-language speakers, see English-language learners

Sex differences, see Gender factors

Social indicator systems, 3, 24, 38-39, 152, 153

Socioeconomic status, 50

poverty, 14, 44

Sociopolitical factors, see Demographic factors;

Political factors

Spanish-speaking students, see Hispanic students

Special studies, 16, 57, 110

Special Study Panel on Education Indicators, 35, 37, 38, 40-41

Standards, general, 19, 20, 27, 126

coordinated system of education indicators, 23, 45

curriculum, 10, 179, 153

mathematics achievement, 58, 59, 73

NAEP history, 12, 15

reporting, standards-based, 2, 7, 25-26, 33, 162, 163-165, 167-168, 171-173, 182, 184

science achievement, 10, 166, 168-171, 178-179

students with disabilities/English-language learners, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113

see also Achievement-level descriptions;

Achievement levels;

Achievement-level

setting;

Performance standards;

Preliminary achievement-level descriptions;

Scoring and scoring criteria

State Collaboratives on Assessment and Standards Project, 40, 46

State government, 19

costs, 78

curriculum reform, 13, 153

evaluation of programs, 11

NAGB and, 25, 65-66

national curriculum, 13

National Governors' Association, 15, 172

standards, 10, 56

students with disabilities/English-language learners, 88, 93, 94, 97, 98, 104, 106, 109, 110

State-level factors and trends, 4, 5, 23, 26, 189

age-vs grade-level testing, 14

coordinated system of education indicators, 45-46

main NAEP, 15, 16, 56, 57, 58, 65-66

reading achievement, 26, 59

sampling, 59-62, 65-66, 68, 83, 85, 194

sociopolitical factors, 10

State NAEP, 14, 15, 16, 45-46, 56, 57, 58, 65-66, 192-193

cost factors, 62, 66, 78, 82, 85

national NAEP vs, 65, 81-84, 85

objectives, 24

reporting, 2, 10, 14, 15, 26, 31, 68, 71

sampling, 59-62, 65-66, 68, 83, 85, 194

streamlining, 81-84

Student-level measures, 40, 42, 43, 45, 46

Students with disabilities, 5, 7-8, 14, 19, 20, 87-113, 134, 191

accommodations/adaptations, 51, 88-89, 91-112 (passim), 191

accountability, 91, 94

achievement levels, 88, 107, 109, 110-111, 112

administration procedures, testing, 91, 92, 99-100

analysis methods, 101, 104, 108, 110

coordinated system of education indicators, 106, 107

core NAEP, 109, 110, 112

cost factors, 8, 107-108, 111

curriculum, 111

data collection, 87, 96, 104, 106, 107, 108, 112, 113

demography of, 89-90

Department of Education role, 106, 109

NCES, 91, 93, 94, 97, 105, 106, 109, 191-192

educational indicators, general, 88, 106, 114

funding, 108, 109

Goals 2000: Educate America Act, 10, 88, 96

historical perspectives, 5, 92-98

identification/classification, 102, 104-105, 106

inclusion, 45, 92, 93, 95, 97-104, 106, 109, 111-113, 191

individualized education plans, 88

interpretive information, 5, 87, 99, 106, 112

large-scale assessments, 88, 89, 91-93, 96, 97, 98, 104, 106-109, 110, 112-113

legislation, 10, 88, 106, 108

NCES role, 91, 93, 94, 97, 105, 106, 109, 191-192

participation, 5, 19, 57, 87-88

performance standards, 87, 89, 94

reading achievement, 94, 110

reporting procedures, 108, 111

sampling, 101, 102, 103, 106, 107

standards, 5, 87, 89, 91, 92, 94, 98-102, 109, 111-113

validity issues, 88, 91, 95, 106, 109-111

see also Learning disabilities

Students with limited English proficiency, see English-language learners

Student/teacher ratios, 44

Summary scores, 1-2, 6

demographic trends, 14

error of measurement, 63

frameworks and assessment development process, 116, 117, 121, 128, 139, 152, 154

Task measures, 4, 12, 192

coordinated system of education indicators, 23, 24, 33, 47, 54

framework and assessment development process, 119, 128, 132, 136, 139, 142-144, 146, 149, 153-154, 160, 219-220

group, 47, 138

streamlining NAEP design, 67, 68, 71, 81, 86

see also Extended-response items

Teacher background variables, 40, 41, 43

Teacher evaluation of NAEP, 77-78, 129-130

Teaching methods, see Pedagogy

Technical Review Panel, 11, 166

Test administration procedures, see Administration procedures, testing

Text analysis, 219-220

textbooks, 43, 49

Third International Mathematics and Science Study, 36-37, 40, 42-44, 46, 48, 110, 169, 174, 188, 259

Time factors

administration schedules, 18, 121, 191, 192-193, 195

assessment development processes, 124, 136, 159, 160

item response time, 13, 62, 99, 128, 130-131

reading passages, 122

reporting of results, 67, 71, 152, 154

sampling, 69

Trend NAEP, 4, 27, 56, 58, 65, 84, 85

achievement levels, 4-5, 15-16, 56, 58, 77, 78-80

cost factors, 78

frameworks, 77-78, 192, 194

main vs, 4-5, 15-16, 57, 58, 63, 64, 65, 73, 77, 78-80, 84, 85

mathematics achievement, 15, 17, 18, 60, 63, 73, 75, 77, 80, 84, 102

new paradigm, 47, 194

race, 44, 79

reading achievement, 15, 17, 18, 59, 63, 73-75, 77, 80, 84

reporting, 73, 163

sampling, 66

science achievement, 15, 17, 18, 60, 63, 80, 84, 259

writing achievement, 15, 17, 18, 59, 63, 80, 84

Trends in Academic Progress, 65

Trial State Assessment, 25, 94, 96-98 (passim), 174

Urban/rural factors

English-language learners, 89-90

Use of NAEP results, 11, 20, 25-31, 39, 114, 181, 256

achievement levels, 2, 31, 171, 176, 180, 182, 184, 246

mass media, 9-10, 12, 25-30, 68, 71, 171

school districts, 104

stakeholders, 3, 25, 80

Validating National Curriculum Indicators, 40, 46

Validity issues, 11, 122, 153, 167-168, 173, 176-177, 182, 189, 221, 226

analysis and scaling models, 67

content, 12

coordinated system of education indicators, 40, 46

students with disabilities/English-language learners, 88, 91, 95, 106, 109-111

Validity Studies Panel, 11

Video techniques, instructional methods analysis, 43, 46, 48, 194

WESTAT, 17, 119

Writing achievement, 7, 59, 65

main NAEP, 15, 17, 65, 80, 84, 192-193

NAEP administration schedule, 18

NAEP mandate, 24

national NAEP, 16, 18, 59

new paradigm NAEP, 84 extended performance, 16

trend NAEP, 15, 17, 18, 59, 63, 80, 84

trend vs main NAEP, 79, 80

see also Core NAEP