The National Academies Press

Currently Skimming:

Index
Pages 261-272

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.

From page 261... ... , 107�108 ARC Implementation Center study, 100, Accuracy, of content analyses, 78�79 105 Achieved curriculum, 38 Askey, Richard, 24, 79�82, 88 Achievement, importance of social class to, Assessment of existing studies, 2�3 110 case studies, 3, 5 Advanced mathematics at the research level, comparative studies, 2�4 13 content analysis, 2, 5, 90�91 Advanced Placement (AP) courses, 52 final report, 5 exams in, 49 synthesis studies, 3 Alternative experimental approaches, 64 Assignment. Read the entire page →
From page 262... ... Bonferroni method, 111 curricula, 15, 20�22, 97, 99�100, 105, 120, 142�143, 145, 149, 152� 153, 156, 158�159, 162�164, 168, C 198 Calculators, allowing during test taking, for elementary school, 21, 29, 169 and the filters, studies of, 142 53�54 Case studies, 28, 30, 60, 167�180. See also for high school, 22, 29, 169 Comparative studies; Content major textbook publishers, 20�21 market studies not useful in evaluating analyses; Synthesis studies assessment of, 3, 5 curricular effectiveness, 28 authors' backgrounds in, 32 for middle school, 21, 29, 169 secrecy with which market share data comments on, 178�180 criteria for inclusion, 168�169 are held, 20 differential impact on different student Community factors, 44 Comparative analyses, 7�8 populations, 172�175 in establishing curricular effectiveness, appropriate statistical tests, 7 8�9 constraints as to generalizability of study, 7 findings, 171 interactions among curricula and disaggregated data, 7, 158, 200 common practices, beliefs, and in establishing curricular effectiveness, 7�8 understandings, 176�177 patterns in findings, 172 extent of implementation fidelity, 7 professional development, 177�178 outcome measures that can be disaggregated, 7 school location, by study type, 33 the studies, 169 random assignment, 7 time management, 178 Comparative curricula, for content analyses, selection of, 74�75 Case studies methodology, 60, 170�171 backing claims by evidence and Comparative research designs, 58�59 argument, 170 Comparative studies, 2�4, 28, 30, 57�58, 96�166 defining the case, 170 "minimally methodologically adequate" assessment of, 2�4 studies, 97, 101�103, 115, 118�119, authors' backgrounds in, 32 "between" comparisons, 157 136�137, 150, 155, 164 replicability of design, 170�171 comparability of samples, 3 revealing mechanisms at play during conclusions from, 164�166 defining, 97 implementation of a curriculum, 171 description of comparative studies triangulation of evidence from multiple database on critical decision points, 104�164 sources, 60 Catalytic programs, 53 an evolving methodology, 96 Chi-square tests, 128, 157 implementation fidelity, 3 Read the entire page →
From page 263... ... "within" comparisons, 157 Criteria for inclusion, of case studies, 168� Comparative studies database, description 169 on critical decision points, 104�164 Critical decision points in comparative Comparativeness, 132 studies, 104�164 Comprehensiveness alternative hypotheses on effectiveness, of content analyses, 78 137�139 of outcome measures, 9 analysis by test type, 148 Conceptions of mathematics, studies of, choosing statistical tests, 127�132, 199 102 commercial materials studies and the Connected Mathematics Project (CMP) , 19, filters, 142 74, 78, 88�89, 99�100, 118�119, content strand, 149�153 121�122, 133, 172, 175, 177 defining the unit of analysis, 112�114, Connoisseurial assessments, 197 128�130, 147 Conservative test scores, 124 equity analysis, 153�158 Contemporary Mathematics in Context experimental or quasi-experimental (Core-Plus) Read the entire page →
From page 264... ... See Everyday Mathematics Design principles, guidelines for, 4 Embedded assessment, 47 Design replicability, 170�171 Enacted curriculum, 38 Read the entire page →
From page 265... ... A., 62 mathematical, 79 Formative assessment, 47 Type I, 62 Framework for evaluating curricular Establishing curricular effectiveness, 4�9 effectiveness, 36�64 case studies, 8�9 evaluation design, measurement, and comparative analyses, 7�8 evidence, 54�64 content analyses, 6�7 guidelines for future evaluations, 4 scientific, 5, 14, 19 implementation components, 43�48 Ethnographic evaluation, 60 intervention strategies, 52�53 Evaluation of curricular effectiveness, 11, measures of student outcomes, 49�51 50, 54�64, 190 primary components, 40�51 accumulation of knowledge and the program components, 40�43 meta-analysis, 61�64 secondary components, 52�54 articulation of program theory, 54�56 systemic factors, 52 controversy surrounding, 204�205 unanticipated influences, 53�54 cost-efficiency, 11 credibility, 11 educator independence, 61 G ethnographic perspectives, 60 Gagne-type hierarchical structure, 82 including representative samples, 155 informativeness, 11 Game theory, 64 selection of research design and Gender, disaggregated data by, 7, 158, 200 Generalizability methodology, 57�60 time elements, 61 associated with mastery of standard validity, 11 algorithms, 160 in comparative analyses, constraints on, Evaluator bias, 138 Everyday Mathematics (EM) , 19, 83, 100, 7 107, 174, 176, 181 impact on probabilities, 146�147 limitations on, 132�134, 140, 200 example of synthesis studies, 181 Existing studies, assessment of, 2�3 results and limitations resulting from Expectations, standardizing, 156�157 design constraints, 132�134, 140 of results to future circumstances, 56, Experimental approaches, 63 alternative, 64 132 randomization to avoid bias, 63 Generic controls, 58 Group work, 175 Experimental vs. Read the entire page →
From page 266... ... (CPMP) , 20, 80�81, 88, Implementation fidelity, 3 100, 107, 123, 129, 175, 177�178 in comparative studies, 7, 114�118, 139 Integrated Mathematics, 22, 66, 87, 180 Implementation of a curriculum Interactive Mathematics Program, 20, development of a community of 91, 100, 108 practitioners for, 185�186 Larson Series, 22 factors undercutting, 138 MATH Connections, 20 mechanisms at play during, 171 Mathematics: Modeling Our World, 20, trustworthiness of, 8�9, 56 86 Indicators, "corruptibility of," 51 Systemic Initiative for Montana Instructional quality and type, 47 Mathematics and Science, 20, 84, "Integrated Mathematics Project," 182 177, 182 Intended curriculum, 38 University of Chicago School Interactive Mathematics Program (IMP) Read the entire page →
From page 267... ... See Multiple Analysis of Mathematics in Context, 20, 74, 78, 89, Variance 182 Market share data, held in secrecy, 20 MathScape, 20 Market studies, not useful in evaluating MathThematics (STEM) , 20 curricular effectiveness, 28 Middle School Mathematics Through Matched comparison groups, 59 Applications Project, 20 Math 65, 82 Middle School Mathematics Comparisons MATH Connections, 20 for Singapore Mathematics, Math K-5, 21 Connected Mathematics Program, Math Trailblazers, 19, 100 and Mathematics in Context, 71, 85 "Mathematical empowerment," rhetoric of, Middle School Mathematics Through 175 Applications Project (MMAP) Read the entire page →
From page 268... ... See National Science Foundation America's Future, 46 National Council of Teachers of Mathematics (NCTM) , 8, 69, 181 O Curriculum and Evaluation Standards for School Mathematics, 69 Open-ended tasks, measures of, 50 Principles and Standards for School "Opportunity to learn," 47, 124, 194 Mathematics 2000, 71, 197 Organization, of content analyses, 82�83 revised standards written by, 74 Orleans-Hanna Algebraic Prognosis Test, standards written by, 12, 52, 98 124 National decline, blaming curricula for, 188 Ortiz-Franco, Luis, 24 National policy makers Outcome measures, 165�166, 259 as decision makers, 1 careful attention to, 126 need for sound evaluation of curricular and forms of disaggregation, 120�127, developments, 11 140 National Research Council (NRC) Read the entire page →
From page 269... ... See Preliminary Scholastic Reliability, of treatment administration, 108 Assessment Test Remedial mathematics activities, 13 Public discourse, 175 Replicability of design, 170�171 Publishers Reporting the data, varied methods of, 50 need for sound evaluation of curricular Research design and methodology, 57�60 developments, 11 case studies, 60 pressures on, 52 comparative designs, 58�59 comparative studies, 57�58 content analyses, 57 Q Resource-oriented perspectives, in content analyses, 7, 44, 92�93 "Quasi-experiments," 58�59 Results, disaggregated by content strands or generic controls, 58 by performance by student longitudinal studies, 58 subgroups, 3 matched comparison groups, 59 Reviewer's expertise, 73 statistically equated control, 58 Reviews available, on curricula programs supported by the NSF, 203 Robinson, Eric, 81�82 R Race/ethnicity, disaggregated data by, 7, 158, 200 S Random assignment, 108 Sample populations, 166 to avoid bias, 63 comparability of, 3 in comparative analyses, 7 size of, 140 studies not using, 108�112 SAT, 49 Randomized experiments, 62 preparation courses for, 52 Randomized field trials, 59 Saxon materials, 98�100, 112, 143, 147, Recommendations, 9�10, 185�205 164 at district and local levels, 10 pedagogical approach, 56, 82, 87, 112, to federal and state agencies and 125 publishers, 9�10, 201�205 Schifter, Deborah, 24, 76 framework and key definitions, 189�190 School boards, as decision makers, 1 regarding quality of the evaluations, School location, by study type, 33�34 188�189 rural area, 34 Read the entire page →
From page 270... ... , 112, 139, 141, Synthesis studies, 28, 30, 180�184 175 assessment of, 3 disaggregated data by, 7, 158, 200 authors' backgrounds in, 32 importance to achievement, 110 examples of, 181�183 Sophistication of content analysis, Systemic Initiative for Montana increasing, 95 Mathematics and Science (SIMMS) Special interest groups, 48 Integrated Mathematics: A Modeling Standardized tests, 49 Approach Using Technology, 20, 84, Standards, for content analyses, selection of, 161, 177, 182 74�75 State accountability systems, 49 State adoption boards T as decision makers, 1 expressed needs or preferences of, 43 t-tests, 127, 157 Teacher data, by study type, 34�35 Statistical significance, 127�132, 199 Statistical tests in comparative studies, 7, expressed needs or preferences of, 43 127�132, 199 volunteer teachers, 35 Teacher effects, 119�120, 140 Analysis of Covariance, 127�128, 157, 166 in comparative studies, 119�120, 140 Analysis of Variance, 127, 166 strong- vs. Read the entire page →
From page 271... ... See Third International Mathematics and Science Study Traditional curricula, 106, 123 V Traditional Practices, 116�117 Validity, curricular validity of measures, 6, Treatment fidelity, impact on probabilities, 143, 147 9, 49, 122, 126, 195 Trustworthiness, of implementation, 8�9, Vivendi, 21 Volunteer teachers, 35 56 Type I errors, 62 W U Wang, Frank, 55 UCSMP. See University of Chicago School Weak-implementing teachers, 116 Wierenga, Timothy, 46 Mathematics Project Units of analysis "Within" comparisons, 157 defining, 112�114, 128�130, 147 Workshops, defining effectiveness, 23�24 Wu, Hung Hsi, 24, 73, 76 impact on probabilities, 140, 146, 165 using the wrong unit, 138 Read the entire page →

From page 261...

... , 107�108 ARC Implementation Center study, 100, Accuracy, of content analyses, 78�79 105 Achieved curriculum, 38 Askey, Richard, 24, 79�82, 88 Achievement, importance of social class to, Assessment of existing studies, 2�3 110 case studies, 3, 5 Advanced mathematics at the research level, comparative studies, 2�4 13 content analysis, 2, 5, 90�91 Advanced Placement (AP) courses, 52 final report, 5 exams in, 49 synthesis studies, 3 Alternative experimental approaches, 64 Assignment.

Read the entire page →

From page 262...

... Bonferroni method, 111 curricula, 15, 20�22, 97, 99�100, 105, 120, 142�143, 145, 149, 152� 153, 156, 158�159, 162�164, 168, C 198 Calculators, allowing during test taking, for elementary school, 21, 29, 169 and the filters, studies of, 142 53�54 Case studies, 28, 30, 60, 167�180. See also for high school, 22, 29, 169 Comparative studies; Content major textbook publishers, 20�21 market studies not useful in evaluating analyses; Synthesis studies assessment of, 3, 5 curricular effectiveness, 28 authors' backgrounds in, 32 for middle school, 21, 29, 169 secrecy with which market share data comments on, 178�180 criteria for inclusion, 168�169 are held, 20 differential impact on different student Community factors, 44 Comparative analyses, 7�8 populations, 172�175 in establishing curricular effectiveness, appropriate statistical tests, 7 8�9 constraints as to generalizability of study, 7 findings, 171 interactions among curricula and disaggregated data, 7, 158, 200 common practices, beliefs, and in establishing curricular effectiveness, 7�8 understandings, 176�177 patterns in findings, 172 extent of implementation fidelity, 7 professional development, 177�178 outcome measures that can be disaggregated, 7 school location, by study type, 33 the studies, 169 random assignment, 7 time management, 178 Comparative curricula, for content analyses, selection of, 74�75 Case studies methodology, 60, 170�171 backing claims by evidence and Comparative research designs, 58�59 argument, 170 Comparative studies, 2�4, 28, 30, 57�58, 96�166 defining the case, 170 "minimally methodologically adequate" assessment of, 2�4 studies, 97, 101�103, 115, 118�119, authors' backgrounds in, 32 "between" comparisons, 157 136�137, 150, 155, 164 replicability of design, 170�171 comparability of samples, 3 revealing mechanisms at play during conclusions from, 164�166 defining, 97 implementation of a curriculum, 171 description of comparative studies triangulation of evidence from multiple database on critical decision points, 104�164 sources, 60 Catalytic programs, 53 an evolving methodology, 96 Chi-square tests, 128, 157 implementation fidelity, 3

Read the entire page →

From page 263...

... "within" comparisons, 157 Criteria for inclusion, of case studies, 168� Comparative studies database, description 169 on critical decision points, 104�164 Critical decision points in comparative Comparativeness, 132 studies, 104�164 Comprehensiveness alternative hypotheses on effectiveness, of content analyses, 78 137�139 of outcome measures, 9 analysis by test type, 148 Conceptions of mathematics, studies of, choosing statistical tests, 127�132, 199 102 commercial materials studies and the Connected Mathematics Project (CMP) , 19, filters, 142 74, 78, 88�89, 99�100, 118�119, content strand, 149�153 121�122, 133, 172, 175, 177 defining the unit of analysis, 112�114, Connoisseurial assessments, 197 128�130, 147 Conservative test scores, 124 equity analysis, 153�158 Contemporary Mathematics in Context experimental or quasi-experimental (Core-Plus)

Read the entire page →

From page 264...

... See Everyday Mathematics Design principles, guidelines for, 4 Embedded assessment, 47 Design replicability, 170�171 Enacted curriculum, 38

Read the entire page →

From page 265...

... A., 62 mathematical, 79 Formative assessment, 47 Type I, 62 Framework for evaluating curricular Establishing curricular effectiveness, 4�9 effectiveness, 36�64 case studies, 8�9 evaluation design, measurement, and comparative analyses, 7�8 evidence, 54�64 content analyses, 6�7 guidelines for future evaluations, 4 scientific, 5, 14, 19 implementation components, 43�48 Ethnographic evaluation, 60 intervention strategies, 52�53 Evaluation of curricular effectiveness, 11, measures of student outcomes, 49�51 50, 54�64, 190 primary components, 40�51 accumulation of knowledge and the program components, 40�43 meta-analysis, 61�64 secondary components, 52�54 articulation of program theory, 54�56 systemic factors, 52 controversy surrounding, 204�205 unanticipated influences, 53�54 cost-efficiency, 11 credibility, 11 educator independence, 61 G ethnographic perspectives, 60 Gagne-type hierarchical structure, 82 including representative samples, 155 informativeness, 11 Game theory, 64 selection of research design and Gender, disaggregated data by, 7, 158, 200 Generalizability methodology, 57�60 time elements, 61 associated with mastery of standard validity, 11 algorithms, 160 in comparative analyses, constraints on, Evaluator bias, 138 Everyday Mathematics (EM) , 19, 83, 100, 7 107, 174, 176, 181 impact on probabilities, 146�147 limitations on, 132�134, 140, 200 example of synthesis studies, 181 Existing studies, assessment of, 2�3 results and limitations resulting from Expectations, standardizing, 156�157 design constraints, 132�134, 140 of results to future circumstances, 56, Experimental approaches, 63 alternative, 64 132 randomization to avoid bias, 63 Generic controls, 58 Group work, 175 Experimental vs.

Read the entire page →

From page 266...

... (CPMP) , 20, 80�81, 88, Implementation fidelity, 3 100, 107, 123, 129, 175, 177�178 in comparative studies, 7, 114�118, 139 Integrated Mathematics, 22, 66, 87, 180 Implementation of a curriculum Interactive Mathematics Program, 20, development of a community of 91, 100, 108 practitioners for, 185�186 Larson Series, 22 factors undercutting, 138 MATH Connections, 20 mechanisms at play during, 171 Mathematics: Modeling Our World, 20, trustworthiness of, 8�9, 56 86 Indicators, "corruptibility of," 51 Systemic Initiative for Montana Instructional quality and type, 47 Mathematics and Science, 20, 84, "Integrated Mathematics Project," 182 177, 182 Intended curriculum, 38 University of Chicago School Interactive Mathematics Program (IMP)

Read the entire page →

From page 267...

... See Multiple Analysis of Mathematics in Context, 20, 74, 78, 89, Variance 182 Market share data, held in secrecy, 20 MathScape, 20 Market studies, not useful in evaluating MathThematics (STEM) , 20 curricular effectiveness, 28 Middle School Mathematics Through Matched comparison groups, 59 Applications Project, 20 Math 65, 82 Middle School Mathematics Comparisons MATH Connections, 20 for Singapore Mathematics, Math K-5, 21 Connected Mathematics Program, Math Trailblazers, 19, 100 and Mathematics in Context, 71, 85 "Mathematical empowerment," rhetoric of, Middle School Mathematics Through 175 Applications Project (MMAP)

Read the entire page →

From page 268...

... See National Science Foundation America's Future, 46 National Council of Teachers of Mathematics (NCTM) , 8, 69, 181 O Curriculum and Evaluation Standards for School Mathematics, 69 Open-ended tasks, measures of, 50 Principles and Standards for School "Opportunity to learn," 47, 124, 194 Mathematics 2000, 71, 197 Organization, of content analyses, 82�83 revised standards written by, 74 Orleans-Hanna Algebraic Prognosis Test, standards written by, 12, 52, 98 124 National decline, blaming curricula for, 188 Ortiz-Franco, Luis, 24 National policy makers Outcome measures, 165�166, 259 as decision makers, 1 careful attention to, 126 need for sound evaluation of curricular and forms of disaggregation, 120�127, developments, 11 140 National Research Council (NRC)

Read the entire page →

From page 269...

... See Preliminary Scholastic Reliability, of treatment administration, 108 Assessment Test Remedial mathematics activities, 13 Public discourse, 175 Replicability of design, 170�171 Publishers Reporting the data, varied methods of, 50 need for sound evaluation of curricular Research design and methodology, 57�60 developments, 11 case studies, 60 pressures on, 52 comparative designs, 58�59 comparative studies, 57�58 content analyses, 57 Q Resource-oriented perspectives, in content analyses, 7, 44, 92�93 "Quasi-experiments," 58�59 Results, disaggregated by content strands or generic controls, 58 by performance by student longitudinal studies, 58 subgroups, 3 matched comparison groups, 59 Reviewer's expertise, 73 statistically equated control, 58 Reviews available, on curricula programs supported by the NSF, 203 Robinson, Eric, 81�82 R Race/ethnicity, disaggregated data by, 7, 158, 200 S Random assignment, 108 Sample populations, 166 to avoid bias, 63 comparability of, 3 in comparative analyses, 7 size of, 140 studies not using, 108�112 SAT, 49 Randomized experiments, 62 preparation courses for, 52 Randomized field trials, 59 Saxon materials, 98�100, 112, 143, 147, Recommendations, 9�10, 185�205 164 at district and local levels, 10 pedagogical approach, 56, 82, 87, 112, to federal and state agencies and 125 publishers, 9�10, 201�205 Schifter, Deborah, 24, 76 framework and key definitions, 189�190 School boards, as decision makers, 1 regarding quality of the evaluations, School location, by study type, 33�34 188�189 rural area, 34

Read the entire page →

From page 270...

... , 112, 139, 141, Synthesis studies, 28, 30, 180�184 175 assessment of, 3 disaggregated data by, 7, 158, 200 authors' backgrounds in, 32 importance to achievement, 110 examples of, 181�183 Sophistication of content analysis, Systemic Initiative for Montana increasing, 95 Mathematics and Science (SIMMS) Special interest groups, 48 Integrated Mathematics: A Modeling Standardized tests, 49 Approach Using Technology, 20, 84, Standards, for content analyses, selection of, 161, 177, 182 74�75 State accountability systems, 49 State adoption boards T as decision makers, 1 expressed needs or preferences of, 43 t-tests, 127, 157 Teacher data, by study type, 34�35 Statistical significance, 127�132, 199 Statistical tests in comparative studies, 7, expressed needs or preferences of, 43 127�132, 199 volunteer teachers, 35 Teacher effects, 119�120, 140 Analysis of Covariance, 127�128, 157, 166 in comparative studies, 119�120, 140 Analysis of Variance, 127, 166 strong- vs.

Read the entire page →

From page 271...

... See Third International Mathematics and Science Study Traditional curricula, 106, 123 V Traditional Practices, 116�117 Validity, curricular validity of measures, 6, Treatment fidelity, impact on probabilities, 143, 147 9, 49, 122, 126, 195 Trustworthiness, of implementation, 8�9, Vivendi, 21 Volunteer teachers, 35 56 Type I errors, 62 W U Wang, Frank, 55 UCSMP. See University of Chicago School Weak-implementing teachers, 116 Wierenga, Timothy, 46 Mathematics Project Units of analysis "Within" comparisons, 157 defining, 112�114, 128�130, 147 Workshops, defining effectiveness, 23�24 Wu, Hung Hsi, 24, 73, 76 impact on probabilities, 140, 146, 165 using the wrong unit, 138

Read the entire page →

← Previous Chapter Skim

This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.

Index Pages 261-272

Index
Pages 261-272