1
Introduction

The National Academies of Sciences, Engineering, and Medicine have, starting in 1959, annually assembled panels of experts—from academia, industry, medicine, and other scientific and engineering communities of practice—to assess the quality and effectiveness of the National Institute of Standards and Technology (NIST) measurements and standards laboratories, of which there are six,¹ as well as the adequacy of the laboratories’ resources. These reviews are conducted under contract at the request of NIST.

STATEMENT OF TASK

The National Academies shall appoint a panel to assess independently the scientific and technical work performed by the NIST Material Measurement Laboratory. The panel will review technical reports and technical program descriptions prepared by NIST staff and will visit the facilities of the Material Measurement Laboratory. Visits will include technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, and discussions with NIST staff. The panel will deliberate findings in closed sessions and will prepare a report summarizing its assessment findings and recommendations.

NIST has requested that the laboratories be assessed against the following broad criteria:

1. Assess the organization’s technical programs.
   1. How does the quality of the research compare to similar world-class research in the technical program areas?
   2. Is the quality of the technical programs adequate for the organization to reach its stated technical objectives? How could it be improved?
2. Assess the portfolio of scientific expertise within the organization.
   1. Does the organization have world-class scientific expertise in the areas of the organization’s mission and program objectives? If not, in what areas should it be improved?
   2. How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
3. Assess the adequacy of the organization’s budget, facilities, equipment, and human resources.
   1. How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
4. Assess the effectiveness by which the organization disseminates its program outputs.
   1. How well are the organization’s research programs driven by stakeholder needs?

1. 2. How effective are the dissemination methods and technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
   3. How well is this organization monitoring stakeholder use and impact of program outputs? How could this be improved?

CONDUCT OF THE ASSESSMENT

In 2023, at the request of the director of NIST, the National Academies formed the Panel on Assessment of the National Institute of Standards and Technology (NIST) Material Measurement Laboratory (the “panel”), having earlier established the statement of task described above.

The panel’s review covered the six divisions and two offices that comprise MML. The panel conducted its review in person on May 16–18, 2023, with a split meeting between Boulder, Colorado, and Gaithersburg, Maryland. The panel was divided into 8 sub-panels. After an opening plenary session with general presentations and discussions, the panel broke out into its sub-panels to work independently, receiving briefings, engaging in discussions, and going on tours. By design, each sub-panel was focused on the division or office that it assessed. There was a final working plenary where the panel deliberated as a whole and then a session for it to confirm its understandings with MML leadership. NIST staff further provided written responses to the panel’s queries.

To accomplish its mission, the panel reviewed the material provided by MML prior to and during the review meeting. The choice of projects to be reviewed was made by MML. The panel applied a largely qualitative approach to the assessment, using the members’ professional experience, expertise, and judgment to conduct the assessment. Given the non-exhaustive nature of the review, the omission in this report of any particular MML project should not be interpreted as a negative reflection on the omitted project. Similarly, the natural variations between the chapters in terms of length, level of detail, and approach convey no message about the quality of work being performed by MML or the information provided to the panel. Each chapter was an independent sub-panel focused on a specific division or office and conveyed the information they believed to be important based on their interactions with the staff of the particular division or office that they assess. It must be noted that this panel’s charge is not to opine on what work MML should be doing. That is guidance provided by another independent NIST advisory committee, the Visiting Committee on Advanced Technology. This panel’s charge is confined to assessing how well MML is doing its work.

Special Considerations

There are two items that need to be addressed to fully understand this report.

Comparison to International Research Efforts

One of the questions in the statement of task (item 1.a) is “How does the quality of the research compare to similar world-class research in the technical program areas?” The panel was impressed with the quality of research conducted at MML and found it to be generally world-class or world-leading. As with all other assessment topics, this is based on the panelists’ individual and cumulative expertise and experience. However, within the time constraints of this project, there was not time to engage in extensive additional research. Therefore, the report might have fewer concrete examples in regard to this item of the statement of task than the others. Any lack of examples should not be taken as a lack of quality.

Safety

Also, the panel makes observations and a recommendation on safety. It was very careful in doing so. Safety is not part of the formal charge to the panel, but MML leadership indicated that they would like to receive any feedback the panel had on this issue. Two panelists—Elsa Reichmanis and Steven Freilich—have laboratory and leadership safety experience, but the panel was not composed to conduct a rigorous safety review (as it was not tasked to do so). Accordingly, the report notes some things that the panel observed that are of concern to it and makes a high-level safety key recommendation to draw attention to the panel’s concerns and suggest a path forward withing the limits of the scope, information available to the committee, and the panel’s collective expertise. Due to these limitations, the panel took a relatively light touch on this matter.

SUMMARY OF THE 2023 TECHNICAL ASSESSMENT OF THE CAPITAL FACILITY NEEDS OF THE NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY REPORT

In February 2023, the National Academies released the report Technical Assessment of the Capital Facility Needs of the National Institute of Standards and Technology (NASEM 2023; hereafter the “Capital Facility Needs report”). The authoring committee was tasked to assess NIST’s facilities and utility infrastructure, review and assess plans and projects to reinvigorate NIST’s facilities and utility infrastructure, the cost estimates for doing so, and the factors that NIST should consider in developing a comprehensive capital strategy for the facilities and utility infrastructure at NIST’s campuses in Boulder, Colorado, and Gaithersburg, Maryland. The committee that authored the 2023 Capital Facilities report engaged with the Department of the Interior, the National Institutes of Health, the U.S. Army Engineer Research and Development Center, and the Johns Hopkins University Applied Physics Laboratory to learn about their methods and metrics for assessing facility conditions and maintaining their facilities.

The condition of NIST’s facilities and utility infrastructure has been a concern since 2002 when the Visiting Committee on Advanced Technology issued a report calling NIST’s facilities condition and the related funding situation “alarming” and “critical.” Over the following 20 years, the Visiting Committee on Advanced Technology returned consistently to this theme with increasingly dire language. Eventually, the conference report accompanying the Consolidated Appropriations Act of 2021 (P.L. 116-260) requested that NIST “contract with an independent entity to develop a report that assesses the comprehensive capital needs of NIST’s campuses.” In response, NIST’s Office of Facilities and Property Management approached the National Academies to conduct a study based on a successful study and report completed for the National Institutes of Health in 2019 (NASEM 2019). The result was the 2023 Capital Facility Needs report (NASEM 2023).

The committee that authored the Capital Facility Needs report visited both the Boulder, Colorado, and Gaithersburg, Maryland, campuses. They discovered that many NIST facilities are inadequate to support the world-leading research that is NIST’s mission. Both the quality and the reliability of power can be problematic, resulting in slowed work, lost work, and unnecessary time spent recalibrating sensitive instruments. Inadequacies in basic environmental controls can result in laboratories that are too hot or cold, too humid or not humid enough, and lack proper vibration insulation. In one 1950s-era Boulder laboratory the gaps between the windows and frames allow dust to blow straight into the laboratory. Roof leaks have destroyed multimillion-dollar pieces of equipment, such as tunneling electron microscopes in both Boulder and Gaithersburg. A water leak in Gaithersburg resulted in permanent damage to the world-leading Kibble balance that tied the standard kilogram to the speed of light. There are many more instances and stories. In all, the committee that authored the 2023 Capital Facility Needs report found that the NIST research staff loses between 10–40 percent of its working time fighting against facility inadequacies, also consuming research money to do so.

That committee concluded that the situation has reached the point where:

• NIST researchers will not be able to continue their world-class research no matter their efforts;
• This is already impacting the ability to recruit and retain staff and the willingness of foreign researchers to do work at NIST;
• At risk is also NIST’s international credibility and influence and its ability to support national security, U.S. international competitiveness, medical therapeutics, and a wide range of other activities upon which users in the U.S. government, industry, and academia rely.

In the course of its work, the authoring committee found that NIST’s internal facility and property management policies are not responsible for this situation. Rather, the cause is more than two decades of erratic, unpredictable, and inadequate funding for NIST’s Construction of Research Facilities budget, which includes facility sustainment, restoration, modernization, and expansion. Exacerbating this problem is congressionally directed pass-through funding for things like building laboratories on university campuses that are not used by NIST. This pass-through funding is not revenue-neutral to NIST, costing staff time and money to administer, draining even more much-needed money from NIST’s facilities coffers.

In short, the committee that authored the 2023 Capital Facility Needs report found that the situation requires serious and sustained attention, particularly from leadership levels above NIST. The authoring committee also endorsed the coordinated recovery plan drafted by NIST’s Office of Facilities and Property Management and recommended its continued refinement and shortening it to complete it in 12 years. Critically, the authoring committee identified the need for significant and sustained funding to address NIST’s facilities and utility shortcomings and bring them to the standard necessary for modern metrology. This funding is the critical piece of the recovery plan. The committee that authored the 2023 Capital Facility Needs report recommends $420 million to $550 million per year in funding for NIST’s Construction of Research Facilities budget over at least 12 years. As shown in Table 1-1, this includes $120 million to $150 million per year for sustainment, capacity, maintenance, and major repairs funding to address the more than $800 million deferred maintenance backlog and bring existing facilities to an acceptable condition and keep them there. It also includes $300 million to $400 million per year over at least 12 years for the construction and major renovations budget to upgrade, renovate, and build the new laboratories with the new capabilities needed to conduct modern metrology research.

The picture is not unremittingly bleak. NIST has already begun to modernize laboratories as its current budget allows. These new laboratories are state of the art and enable the cutting-edge, world-leading research that is NIST’s mission. As an example, one NIST research group—after waiting 18 months to be relocated into a new, modern laboratory—won the 2021 Physics World Breakthrough of the Year award for a previously unprecedented demonstration of the quantum entanglement of micro-resonators. NIST’s staff is world-class and capable of producing amazing results, results that will serve the nation and inspire the next generations of researchers, provided they are given the facilities and tools needed to do their work.

TABLE 1-1 Overview of NIST Facility and Infrastructure Funding Needs

NOTE: CRF funding is the sum of CMR and SCMMR funding.

SOURCE: NIST (2022).

STRUCTURE OF THIS REPORT

This report opens with this introductory chapter, followed by an overview of MML. Each division and office of MML is then assessed in its own chapter. The structure within each of these chapters is aligned with the statement of task presented above to aid the reader in understanding the panel’s assessment. Within the common structure of each chapter, each sub-panel had the flexibility to implement a substructure that it believed made the most sense for the information that it wanted to convey. This results in different substructures among the chapters. After the assessment chapters is a chapter that presents the recommendations from the fiscal year (FY) 2020 MML assessment report (NASEM 2021) and MML’s responses to those recommendations. The final chapter presents the recommendations from this report in one place for ease of reference. Any issue that the panel believed applies to MML as a whole is addressed with a key recommendation. These are presented in the Summary, in the chapters in which they appear, and in the final recommendation summary chapter. The structure of the report chapters lays out thus:

• Chapter 1: Introduction
• Chapter 2: Overview of the Material Measurement Laboratory
• Chapter 3: Applied Chemicals and Materials Division
• Chapter 4: Biosystems and Biomaterials Division
• Chapter 5: Biomolecular Measurement Division
• Chapter 6: Chemical Sciences Division
• Chapter 7: Materials Measurement Science Division
• Chapter 8: Materials Science and Engineering Division
• Chapter 9: Office of Data and Informatics
• Chapter 10: Office of Reference Materials
• Chapter 11: Material Measurement Laboratory’s Responses to the Findings and Recommendations of the Fiscal Year 2020 Assessment Report
• Chapter 12: Overarching Themes, Key Recommendations, and Chapter-Specific Recommendations from This Assessment

REFERENCES