National Academies Press: OpenBook
« Previous: Summary
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×

1

Introduction

What is now called the National Institute of Standards and Technology (NIST) was founded by Congress in 1901 as the National Bureau of Standards. The founding charge was to establish a measurement infrastructure to enhance U.S. industrial competitiveness. NIST is now part of the Department of Commerce with a stated mission “to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.”1 The Communications Technology Laboratory (CTL) was established in 2014 as a research and development organization to unite NIST’s ongoing wireless communications efforts. CTL builds on NIST’s long history of radio science and measurement.

Since 1959, the National Academies of Science, Engineering, and Medicine have been contracted to provide assessments of NIST’s laboratory programs. The National Academies’ assessments are conducted by panels of experts from industry, academia, and other scientific and engineering communities of practice, and each addresses a statement of task provided by NIST.2 In 2022, at the request of the director of NIST, the National Academies formed the 2022 Panel on Review of the National Institute of Standards and Technology’s Communications Technology Laboratory with the following statement of task.

STATEMENT OF TASK

The National Academies shall form a panel of experts who shall perform an independent technical assessment of the quality of NIST’s CTL. The panel shall hold a virtual site visit to the CTL in Boulder, Colorado, to conduct an in-depth technical assessment of CTL’s work. The panel will then use any materials provided by NIST before the remote site visit, and the results of that site visit, to draft a report that reflects the expert consensus of the panel. The assessment will address the following factors at the request of the NIST Director:

___________________

1 National Institute of Standards and Technology, 2022, “About NIST,” https://www.nist.gov/about-nist, accessed September 16, 2022.

2 A list with links to past assessments can be found at https://www.nist.gov/director/national-academies-national-research-council.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×
  1. Assess the organization’s technical programs.
    • How does the quality of the research compare to similar world-class research in the technical program areas?
    • Is the quality of the technical programs adequate for the organization to reach its technical objectives? How could it be improved?
  2. Assess the portfolio of scientific expertise within the organization.
    • Does the organization have world-class scientific expertise in the areas of the organization’s mission and program objectives? If not, in what areas should it be improved?
    • How well does the organization’s scientific expertise support the organization’s technical programs and the organization’s ability to achieve its stated objectives?
  3. Assess the adequacy of the organization’s facilities, equipment, and human resources.
    • How well do the facilities, equipment, and human resources support the organization’s technical programs and its ability to achieve its stated objectives? How could they be improved?
  4. Assess the effectiveness by which the organization disseminates its program outputs.
    • How well are the organization’s research programs driven by stakeholder needs?
    • How effective are the dissemination methods and technology transfer mechanisms used by the organization? Are these mechanisms sufficiently comprehensive?
    • How well is this organization monitoring stakeholder use and impact of program outputs? How could this be improved?

The CTL director asked the panel to review work in six research focus areas: (1) Core Network Technologies, (2) Fundamental Electromagnetic Technologies and Standards, (3) Next-Generation Wireless Systems, (4) Public Safety Communications Research, (5) Smart Infrastructure and Manufacturing, and (6) Spectrum Sharing and Sensing.

CONDUCT OF THE REVIEW

This assessment is the third National Academies review of CTL, with the previous reviews taking place in 2015 and 2019.3 In contrast to the two previous reviews that involved visits to the main CTL site in Boulder, Colorado, the information gathering for the 2022 report was done solely remotely due to the ongoing COVID-19 pandemic. This changed the nature of the panel’s information-gathering process in key ways. For example, during the 2019 review, the panelists spent 2.5 days on site at CTL in Boulder, Colorado, providing, in addition to the formal sessions, many opportunities to interact with CTL personnel informally during breaks, lunches, and a dinner arranged specifically to provide additional interaction opportunities. In addition, the onsite review meeting enabled the panelists to tour the Boulder facilities and see demonstrations of some of the laboratory capabilities. This facilitated the evaluation of the facilities, equipment, and capabilities. The lack of these opportunities during this review limited the information that this panel could collect to conduct its work.

The information gathering done by this panel formally began with a review meeting consisting of 2.5 days of online sessions with CTL. The first 2 days largely consisted of CTL presentations with limited time for questions after each presentation. On the third day, there was a question and answer session with time allocated to each focus area. After the initial meeting, the panel provided lists of additional questions to CTL in each of the focus areas. CTL provided written responses to the questions, and there were 1.5-hour online follow-up meetings in five of the six research focus areas to discuss the responses and any other questions the panelists had.4

___________________

3 Ibid.

4 The panelists in one of the research focus areas felt that a follow-up meeting was not necessary.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×

The panel used the information that CTL provided before and during the online sessions to conduct the assessment using a largely qualitative approach. Time constraints limited the amount of information gathering possible, so the panel focused on the research that the leadership of CTL chose to present. The panel members assessed the factors detailed in the statement of task, to the extent they could, using their professional judgment, their collective knowledge, and experience. Also, this report assesses only the work that CTL is currently doing and any opportunities and challenges related to that work. Questions of what work CTL or any other NIST laboratory should be doing fall under the purview of a separate independent NIST-sponsored body, the Visiting Committee on Advanced Technology. As such, opining on whether CTL is doing the right kinds of research is outside the purview of the panel.

A REMOTE-ONLY REVIEW

Overall, the online format impeded this review and had the following impacts:

  • It limited the interactions between the panelists and CTL personnel at the meetings. During the meetings, most of the information was conveyed by area group leaders. Although the leaders were best positioned to summarize the content, especially given the limited time, the panel missed out on interacting with those doing the work.
  • In addition, the large group setting of the meetings with various levels of personnel may have further limited the panel’s ability to gather information from the researchers working most directly in the research.
  • The online format also severely impacted the panel’s ability to assess the facilities and equipment, and personnel. The panel members did their best to extract facility information where they could, with varying success.
  • It also impacted how well the panelists could process the information being received. The information-gathering process required very long days in front of the computer, working across several time zones.
  • It also limited the panelists’ interactions with each other and the networking that is often the best part of this kind of volunteering.

On the positive side, the online format enabled the recording of the presentations along with the sidebar conversations and questions in the chat. This allowed for an enduring record of the meetings and was helpful in drafting the report.

Recommendation 1-1: If at all possible, the leadership of NIST’s laboratories should ensure that all future assessments have an in-person component to allow the panels to see the facilities and equipment and to interact directly—both formally and informally—with the researchers.

THIS REPORT

This report is structured as follows:

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×

This report includes panel recommendations that specify “who should do what” to address any areas that the panel believes warrant action. The panel underpins the recommendations with salient examples of programs and projects that are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to projects and programs. Key recommendations are those that the panel considers especially worthy of attention.

Each chapter was led by designated chapter teams. While the panel attempted to apply a uniform organization to each chapter, not every chapter team felt that this was possible. Different teams felt that they received different levels of information and wrote their chapters accordingly.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×
Page 4
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×
Page 5
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×
Page 6
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2023. 2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/26778.
×
Page 7
Next: 2 Overview of the Communications Technology Laboratory »
2022 Assessment of the National Institute of Standards and Technology's Communications Technology Laboratory Get This Book
×
Buy Paperback | $23.00 Buy Ebook | $18.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

At the request of the director of the National Institute of Standards and Technology, this report assesses the management of Standards and Technology Communications Technology Laboratory (CTL), focusing on the work, facilities, equipment, personnel, portfolios of scientific expertise, and effective dissemination of the results.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!