Skip to main content

Currently Skimming:

9 Technological Infrastructure
Pages 79-86

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 79...
... CURRENT COSTS Technology is covered by two contracts in the NAEP Alliance, one for the web and the other for platform development.1 The estimated annual average cost is $10.2 million for the web contract and $19.2 million for 1 NCES response to Q33: The "web/technology development, operations and management" contract covers the following activities: "Develops, implements, and supports Internet-related applications and services; identifies and deploys emerging technologies and new products to improve NAEP's web and other computer-based products and services; monitors compliance with all NCES web requirements, and ensures timely release of quality products and services using Web technologies." The "NAEP platform development" contract covers the following activities: "Develops NAEP assessment delivery platform utilizing a state-of-the-art, ageappropriate user interface (UI) and overall user experience (UX)
From page 80...
... This situation can make it prohibitively expensive and slow to implement psychometric innovations that are based on response data, such as adaptive testing, automated scoring, adaptive reporting, and other operational implementations of advanced algorithms. By reducing the manual merging, integration, and analysis of data files, NAEP can become more efficient and reduce the effort needed for administration. Contemporary data architectures provide an elegant and effective solution to these issues, with secured authentication systems providing access application programming interface endpoints in a standardized manner by any technology application.
From page 81...
... These common industry practices include • Standards-based development: the use of industry standards, such as IMS QTI3 for item structure and xAPI4 for process data format, increases the speed of development and enables integration with other systems and analysis.  • Cloud-based technology systems: the use of cloud native platforms, such as Amazon Web Services or Microsoft Azure, can provide the foundation for increased interoperability in a distributed system, as well as improved services and reduced cost. This is likely to be true for NAEP, for which the small annual assessment window makes it attractive to have an infrastructure system that can be "turned off" when not in use.  • Federated architecture: an assessment architecture based on a "sys tem of systems" principle, linked together through these protocols and data security standards, enables new systems and technologies to be added as they become available.  These new and now common practices suggest important staffing considerations in developing such systems.
From page 82...
... This new system includes not only an assessment delivery platform application for students, but also a library of reusable item components, a "data lake" to store all current and historical NAEP data, and mechanisms for data access with a commitment to extensibility, reusability, and other contemporary software design principles. The current contract goes through 2024 and includes a field test for online administration using NAEP-owned devices in 2023 and a proof-of-concept study for delivering NAEP on school-owned devices in 2024.6 An operational system is a deliverable of the current contract, and Next-Gen eNAEP will be used for both the pilot and operational NAEP administrations in 2024.7 After 2024, the system will be used to deliver NAEP on school-owned devices in a field test in 2025 and then for an operational test in 2026.
From page 83...
... Given the dynamism in the field of software development, it is likely that the options available to NCES for both building and buying the relevant components of eNAEP are substantially different than they were when the decision was made to develop the system in-house. Fortunately, the platform development contract includes research and business analysis of other technologies to "stay continually abreast of the latest trends and innovations in large-scale assessment and education technology."9 As part of this work, it is important to be clear that many vendors have successfully administered K–12 summative assessments using platforms that are based on the kinds of software tools and standards specified by NAEP.
From page 84...
... The NAEP Alliance member selected to perform this activity is a global leader in psychometric research and development and is not known in the field for development of production enterprise software applications and assessment technology platforms. The planning documents provided to the panel reference best-of-breed approaches to software development using agile planning, cloud-native technologies, extensible and modular code, and other approaches.
From page 85...
... RECOMMENDATION 9-3: The National Center for Education Statis tics should seek expert guidance from enterprise application developers and educational technologists who understand assessment technology platforms to evaluate the reasonability of the projected costs for the development of Next-Gen eNAEP. 12 NCES response to Q73c.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.