The National Academies Press

Currently Skimming:

Pages 131-146

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.

From page 131... ... G-1 APPENDIX G ROUND 1 EVALUATION – RATINGS, SCORES, AND STATISTICS Twenty members of ACRP 02-09 Panel and the project team evaluated 5 model design concepts based on the modified Pugh Matrix. Read the entire page →
From page 132... ... G-2 TABLE G-1 Round 1 Scores - Agency Acceptance Performance Evaluation Criteria Weight Agency Acceptance 0.30 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 3 3 2 2 Evaluator 2 3 5 2 3 3 Evaluator 3 3 3 3 3 4 Evaluator 4 3 3 2 1 1 Evaluator 5 3 2 1 1 2 Evaluator 6 3 4 3 2 2 Evaluator 7 3 2 4 1 2 Evaluator 8 3 5 2 1 2 Evaluator 9 3 2 1 2 2 Evaluator 10 3 2 2 2 2 Evaluator 11 3 2 2 2 2 Evaluator 12 3 3 2 1 1 Evaluator 13 3 1 2 1 2 Evaluator 14 3 5 5 2 2 Evaluator 15 3 3 2 2 3 Evaluator 16 3 2 2 2 2 Evaluator 17 3 2 2 1 2 Evaluator 18 3 2 3 2 2 Evaluator 19 3 4 3 1 2 Evaluator 20 3 2 2 2 2 3.00 2.50 2.00 2.00 2.00 3.00 2.00 2.00 1.00 2.00 3.00 3.25 3.00 2.00 2.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 133... ... G-3 TABLE G-2 Round 1 Scores - Technical Feasibility Performance Evaluation Criteria Weight Technical Feasibility 0.20 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 4 2 3 3 Evaluator 2 3 5 3 4 4 Evaluator 3 3 3 4 5 4 Evaluator 4 3 3 3 2 4 Evaluator 5 3 2 2 2 2 Evaluator 6 3 4 2 3 2 Evaluator 7 3 2 1 4 2 Evaluator 8 3 3 4 3 3 Evaluator 9 3 3 1 3 2 Evaluator 10 3 4 3 3 3 Evaluator 11 3 4 2 2 3 Evaluator 12 3 3 3 3 2 Evaluator 13 3 3 1 1 1 Evaluator 14 3 5 3 4 1 Evaluator 15 3 4 3 4 3 Evaluator 16 3 5 2 3 3 Evaluator 17 3 3 3 3 3 Evaluator 18 3 3 2 2 2 Evaluator 19 3 5 4 4 3 Evaluator 20 3 4 2 2 2 3.00 3.50 2.50 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 4.00 3.00 4.00 3.00 Design Concepts Median 1st Quartile 3rd Quartile Evaluator D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 134... ... G-4 TABLE G-3 Round 1 Scores - Analytical Proficiency Performance Evaluation Criteria Weight Analytical Proficiency 0.20 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 3 3 3 2 Evaluator 2 3 3 3 4 4 Evaluator 3 3 1 3 4 3 Evaluator 4 3 1 3 3 3 Evaluator 5 3 2 4 3 4 Evaluator 6 3 2 3 2 3 Evaluator 7 3 2 4 4 2 Evaluator 8 3 4 2 3 3 Evaluator 9 3 3 4 4 4 Evaluator 10 3 2 3 3 3 Evaluator 11 3 2 3 3 3 Evaluator 12 3 3 3 2 2 Evaluator 13 3 1 2 4 1 Evaluator 14 3 4 3 3 3 Evaluator 15 3 3 3 4 3 Evaluator 16 3 2 2 4 3 Evaluator 17 3 2 3 3 2 Evaluator 18 3 3 2 3 3 Evaluator 19 3 3 2 4 2 Evaluator 20 3 2 4 2 2 3.00 2.00 3.00 3.00 3.00 3.00 2.00 2.75 3.00 2.00 3.00 3.00 3.00 4.00 3.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00Rating Al te rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 135... ... G-5 TABLE G-4 Round 1 Scores - Scalability Performance Evaluation Criteria Weight Scalability 0.15 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 4 3 3 3 Evaluator 2 3 4 3 3 4 Evaluator 3 3 2 2 3 5 Evaluator 4 3 2 4 4 4 Evaluator 5 3 3 4 3 4 Evaluator 6 3 3 4 3 2 Evaluator 7 3 3 4 4 2 Evaluator 8 3 2 4 2 2 Evaluator 9 3 4 4 4 4 Evaluator 10 3 4 4 4 4 Evaluator 11 3 3 3 4 4 Evaluator 12 3 3 3 3 3 Evaluator 13 3 3 4 4 2 Evaluator 14 3 5 3 5 4 Evaluator 15 3 4 3 4 3 Evaluator 16 3 4 4 4 4 Evaluator 17 3 2 3 3 3 Evaluator 18 3 2 3 3 3 Evaluator 19 3 4 5 3 4 Evaluator 20 3 3 4 4 2 3.00 3.00 4.00 3.50 3.50 3.00 2.75 3.00 3.00 2.75 3.00 4.00 4.00 4.00 4.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 136... ... G-6 TABLE G-5 Round 1 Scores - Responsiveness Performance Evaluation Criteria Weight Responsiveness 0.10 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 4 2 2 1 Evaluator 2 3 3 2 3 3 Evaluator 3 3 2 2 4 3 Evaluator 4 3 2 2 2 3 Evaluator 5 3 2 2 4 2 Evaluator 6 3 3 4 3 3 Evaluator 7 3 3 4 4 3 Evaluator 8 3 2 3 2 2 Evaluator 9 3 2 3 2 3 Evaluator 10 3 3 3 2 3 Evaluator 11 3 2 3 1 3 Evaluator 12 3 2 3 1 1 Evaluator 13 3 3 4 2 3 Evaluator 14 3 4 3 1 3 Evaluator 15 3 3 2 2 3 Evaluator 16 3 3 4 4 4 Evaluator 17 3 2 2 1 2 Evaluator 18 3 3 2 2 3 Evaluator 19 3 2 3 3 3 Evaluator 20 3 2 4 2 2 3.00 2.50 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.25 3.00 3.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 137... ... G-7 TABLE G-6 Round 1 Scores - International Credibility Performance Evaluation Criteria Weight International Credibility 0.05 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 3 4 3 4 Evaluator 2 3 4 3 4 5 Evaluator 3 3 1 4 5 5 Evaluator 4 3 3 4 4 4 Evaluator 5 3 3 4 5 4 Evaluator 6 3 4 2 3 2 Evaluator 7 3 3 3 3 3 Evaluator 8 3 4 2 1 4 Evaluator 9 3 2 3 3 4 Evaluator 10 3 3 3 3 3 Evaluator 11 3 4 3 4 4 Evaluator 12 3 3 3 1 2 Evaluator 13 3 2 3 3 3 Evaluator 14 3 4 3 5 5 Evaluator 15 3 3 3 4 4 Evaluator 16 3 4 3 2 4 Evaluator 17 3 2 3 2 4 Evaluator 18 3 4 4 3 3 Evaluator 19 3 4 3 3 4 Evaluator 20 3 3 4 4 5 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 138... ... G-8 TABLE G-7 Round 1 Scores - Cost Implications Performance Evaluation Criteria Weight Cost Implications 1.00 Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 2 4 3 4 Evaluator 2 3 2 3 2 2 Evaluator 3 3 2 3 3 2 Evaluator 4 3 2 4 4 4 Evaluator 5 3 4 3 3 3 Evaluator 6 3 1 2 5 4 Evaluator 7 3 4 5 5 5 Evaluator 8 3 2 3 5 3 Evaluator 9 3 2 4 4 4 Evaluator 10 3 2 4 4 4 Evaluator 11 3 2 4 4 3 Evaluator 12 3 4 4 5 4 Evaluator 13 3 3 4 4 3 Evaluator 14 3 1 2 3 1 Evaluator 15 3 2 4 5 3 Evaluator 16 3 2 2 4 5 Evaluator 17 3 2 3 4 4 Evaluator 18 3 4 2 4 5 Evaluator 19 3 3 3 4 3 Evaluator 20 3 2 5 4 5 3.00 2.00 3.50 4.00 4.00 3.00 2.00 3.00 3.75 3.00 3.00 3.00 4.00 4.25 4.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.001.502.002.503.003.504.004.505.00 Rating A lte rn at iv e Note that Rating scale is reversed. Median 3rd Quartile 1st Quartile Read the entire page →
From page 139... ... G-9 TABLE G-8 Round 1 Scores - Performance Score Performance Score Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3.00 3.45 2.75 2.60 2.35 Evaluator 2 3.00 4.20 2.60 3.45 3.65 Evaluator 3 3.00 2.25 3.00 3.80 3.90 Evaluator 4 3.00 2.35 2.80 2.30 2.80 Evaluator 5 3.00 2.20 2.50 2.40 2.80 Evaluator 6 3.00 3.35 3.00 2.50 2.30 Evaluator 7 3.00 2.30 3.35 3.05 2.15 Evaluator 8 3.00 3.60 2.80 2.05 2.50 Evaluator 9 3.00 2.70 2.35 2.95 2.90 Evaluator 10 3.00 2.85 2.85 2.75 2.85 Evaluator 11 3.00 2.65 2.50 2.50 2.90 Evaluator 12 3.00 2.90 2.70 1.90 1.75 Evaluator 13 3.00 1.95 2.35 2.25 1.75 Evaluator 14 3.00 4.65 3.60 3.10 2.55 Evaluator 15 3.00 3.35 2.60 3.20 3.05 Evaluator 16 3.00 3.10 2.55 3.10 3.00 Evaluator 17 3.00 2.20 2.60 2.15 2.45 Evaluator 18 3.00 2.60 2.55 2.40 2.50 Evaluator 19 3.00 3.80 3.30 2.80 2.70 Evaluator 20 3.00 2.60 3.00 2.40 2.15 3.00 2.78 2.73 2.55 2.63 3.00 2.34 2.55 2.38 2.34 3.00 3.38 3.00 3.06 2.903rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00Performance Score A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 140... ... G-10 TABLE G-9 Round 1 Scores - Cost Score Cost Score Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 3 2 4 3 4 Evaluator 2 3 2 3 2 2 Evaluator 3 3 2 3 3 2 Evaluator 4 3 2 4 4 4 Evaluator 5 3 4 3 3 3 Evaluator 6 3 1 2 5 4 Evaluator 7 3 4 5 5 5 Evaluator 8 3 2 3 5 3 Evaluator 9 3 2 4 4 4 Evaluator 10 3 2 4 4 4 Evaluator 11 3 2 4 4 3 Evaluator 12 3 4 4 5 4 Evaluator 13 3 3 4 4 3 Evaluator 14 3 1 2 3 1 Evaluator 15 3 2 4 5 3 Evaluator 16 3 2 2 4 5 Evaluator 17 3 2 3 4 4 Evaluator 18 3 4 2 4 5 Evaluator 19 3 3 3 4 3 Evaluator 20 3 2 5 4 5 3.00 2.00 3.50 4.00 4.00 3.00 2.00 3.00 3.75 3.00 3.00 3.00 4.00 4.25 4.003rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 1.001.502.002.503.003.504.004.505.00 Cost Score A lte rn at iv e Note that Cost Score scale is reversed. Median 3rd Quartile 1st Quartile Read the entire page →
From page 141... ... G-11 TABLE G-10 Round 1 Scores - Value Score Value Score Datum Current Preferred Design Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Softw are Alternative #4 Build on EC IMAGINE Evaluator 1 1.00 1.73 0.69 0.87 0.59 Evaluator 2 1.00 2.10 0.87 1.73 1.83 Evaluator 3 1.00 1.13 1.00 1.27 1.95 Evaluator 4 1.00 1.18 0.70 0.58 0.70 Evaluator 5 1.00 0.55 0.83 0.80 0.93 Evaluator 6 1.00 3.35 1.50 0.50 0.58 Evaluator 7 1.00 0.58 0.67 0.61 0.43 Evaluator 8 1.00 1.80 0.93 0.41 0.83 Evaluator 9 1.00 1.35 0.59 0.74 0.73 Evaluator 10 1.00 1.43 0.71 0.69 0.71 Evaluator 11 1.00 1.33 0.63 0.63 0.97 Evaluator 12 1.00 0.73 0.68 0.38 0.44 Evaluator 13 1.00 0.65 0.59 0.56 0.58 Evaluator 14 1.00 4.65 1.80 1.03 2.55 Evaluator 15 1.00 1.68 0.65 0.64 1.02 Evaluator 16 1.00 1.55 1.28 0.78 0.60 Evaluator 17 1.00 1.10 0.87 0.54 0.61 Evaluator 18 1.00 0.65 1.28 0.60 0.50 Evaluator 19 1.00 1.27 1.10 0.70 0.90 Evaluator 20 1.00 1.30 0.60 0.60 0.43 1.00 1.31 0.77 0.63 0.71 1.00 1.01 0.67 0.57 0.58 1.00 1.69 1.03 0.78 0.943rd Quartile Design Concepts Evaluator Median 1st Quartile D at um 0 1 2 3 4 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Value Score A lte rn at iv e Median 3rd Quartile 1st Quartile Read the entire page →
From page 142... ... G-12 Criteria Weight Rating Agency Acceptance 0.30 Technical Feasibility 0.20 Analytical Proficiency 0.20 Scalability 0.15 Responsiveness 0.10 International Credibility 0.05 Cost Implications 1.00 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Rating A lte rn at iv e 0 1 2 3 4 1.001.502.002.503.003.504.004.505.00 Rating A lte rn at iv e Note that Rating scale is reversed. Figure G-1. Read the entire page →
From page 143... ... G-13 Performance Score Cost Score Value Score Scores 0 1 2 3 4 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Performance Score Al te rn at iv e 0 1 2 3 4 1.001.502.002.503.003.504.004.505.00 Cost Score Al te rn at iv e Note that Cost Score scale is reversed. 0 1 2 3 4 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 Value Score Al te rn at iv e Figure G-2. Read the entire page →
From page 144... ... G-14 0 5 10 15 1 2 3 4 5 R at in g Fr eq ue nc y Rating Technical Feasibility Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Sof tware Alternative #4 Build on EC IMAGINE Figure G-4. Distribution of ratings for performance criterion – technical feasibility. Read the entire page →
From page 145... ... G-15 0 5 10 15 1 2 3 4 5 R at in g Fr eq ue nc y Rating Scalability Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Sof tware Alternative #4 Build on EC IMAGINE Figure G-6. Distribution of ratings for performance criterion – scalability 0 5 10 15 1 2 3 4 5 R at in g Fr eq ue nc y Rating Responsiveness Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Sof tware Alternative #4 Build on EC IMAGINE FigureF-7. Read the entire page →
From page 146... ... G-16 0 5 10 15 1 2 3 4 5 R at in g Fr eq ue nc y Rating International Credibility Alternative #1 Build on AEDT Alternative #2 Build on Existing Simulation Models Alternative #3 Federal Adoption of Commercial Sof tware Alternative #4 Build on EC IMAGINE Figure G-8. Distribution of ratings for performance criterion – international credibility. Read the entire page →

From page 131...

... G-1 APPENDIX G ROUND 1 EVALUATION – RATINGS, SCORES, AND STATISTICS Twenty members of ACRP 02-09 Panel and the project team evaluated 5 model design concepts based on the modified Pugh Matrix.