Ontologies in the
Behavioral Sciences

Accelerating Research and the
Spread of Knowledge

Committee on Accelerating Behavioral Science
through Ontology Development and Use

Robert M. Kaplan and Alexandra S. Beatty, Editors

Board on Behavioral, Cognitive, and Sensory Sciences

Division on Behavioral and Social Sciences and Education

A Consensus Study Report of

THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

This activity was supported by contracts between the National Academy of Sciences and the American Psychological Association, Association for Psychological Science, Federation of Associations in Behavioral & Brain Sciences, National Cancer Institute, National Institute on Aging, National Library of Medicine, National Science Foundation (1729167), and National Institutes of Health Office of Behavioral and Social Sciences Research (OBSSR) (HHSN263201800029I/75N98020F00010). Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.

International Standard Book Number-13: 978-0-309-27731-0
International Standard Book Number-10: 0-309-27731-0
Digital Object Identifier: https://doi.org/10.17226/26464

Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Copyright 2022 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. Ontologies in the Behavioral Sciences: Accelerating Research and the Spread of Knowledge. Washington, DC: The National Academies Press. https://doi.org/10.17226/26464.

COMMITTEE ON ACCELERATING BEHAVIORAL SCIENCE THROUGH ONTOLOGY DEVELOPMENT AND USE

ROBERT M. KAPLAN¹ (Chair), Department of Medicine, Primary Care, and Population Health, Stanford University

DEMBA BA, Brain Science Initiative, Harvard University

LISA FELDMAN BARRETT, College of Science, Northeastern University

JIANG BIAN, Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida

KATY BÖRNER, Luddy School of Informatics, Computing, and Engineering, Indiana University

BRUCE F. CHORPITA, Department of Psychology, University of California, Los Angeles

DAVID DANKS, Halıcıoğlu Data Science Institute and Department of Philosophy, University of California, San Diego

KARINA W. DAVIDSON, Feinstein Institutes for Medical Research, Northwell Health

RANDALL W. ENGLE,^2,3 School of Psychology, Georgia Institute of Technology

CATHERINE A. HARTLEY, Department of Psychology and Center for Neural Science. New York University

MARK A. MUSEN,¹ Center for Biomedical Informatics Research, Stanford University

VIMLA L. PATEL, Center for Cognitive Studies in Medicine and Public Health, New York Academy of Medicine

FRANK PUGA, School of Nursing, University of Alabama at Birmingham

CARLA SHARP, Department of Psychology, University of Houston

TIMOTHY J. STRAUMAN, Institute for Brain Sciences, Duke University

CUI TAO, School of Biomedical Informatics, University of Texas Health Center at Houston

JAMES F. WOODWARD, Department of History and Philosophy of Science, University of Pittsburgh

ALEXANDRA S. BEATTY, Study Director, Senior Program Officer

TINA M. WINTERS, Associate Program Officer

J. ASHTON BULLOCK, Senior Program Assistant

___________________

¹ Member, National Academy of Medicine.

² Member, National Academy of Science.

³ Resigned from committee May 2021.

BOARD ON BEHAVIORAL, COGNITIVE, AND SENSORY SCIENCES

TERRIE E. MOFFITT¹ (Chair), Duke University

RICHARD N. ASLIN,² Yale University

JOHN BAUGH, Washington University in St. Louis

WILSON S. GEISLER,² University of Texas at Austin

MICHELE GELFAND,² University of Maryland, College Park

ULRICH MAYR, University of Oregon

KATHERINE L. MILKMAN, University of Pennsylvania

ELIZABETH A. PHELPS, Harvard University

DAVID E. POEPPEL, New York University

STACEY SINCLAIR, Princeton University

TIMOTHY J. STRAUMAN, Duke University

SAMANTHA CHAO, Acting Director

___________________

¹ Member, National Academy of Medicine.

² Member, National Academy of Sciences.

Preface

There are few people whose lives have not been touched in some way by behavioral science research. Topics of study in this domain range from developmental and abnormal psychology to political science, sociology, and behavioral economics. Findings from this work guide diagnosis and treatments of mental disorders, shape policy, and help people make sense of individual behavior and individuals’ relationship to the world around them. These disciplines are flourishing in many ways, but progress—as in any science discipline—requires that scientists share a common vocabulary. Over the last few decades, incentives to innovate in the behavioral sciences have resulted in a proliferation of theories, constructs, and measures, which has led to a range of problems in both building and applying knowledge. The link between these challenges for the behavioral sciences and the comparative lack of development of ontologies in these fields has attracted increasing attention. Ontologies—systems for assigning definitions to the concepts that are important in a particular domain—sound arcane but are in fact fundamental to scientific progress. Other scientific domains have made greater progress in establishing unified languages and shared conceptualizations, and this project’s sponsors, among others, recognized that improved ontologies will be critical to accelerating progress in the behavioral sciences. We thank the project’s sponsors: at the National Institutes of Health, the Office of Behavioral and Social Sciences Research, the National Institute on Aging, the National Library of Medicine, and the National Cancer Institute; the National Science Foundation; the American Psychological Association; the Association for Psychological Science; and the Federation of Associations in Behavioral and Brain Sciences. They collaborated in developing the

statement of task for the study and also provided valuable perspectives to the committee as we began our work.

The committee also gratefully acknowledges the support and contributions of many individuals. They include the scholars who participated in our two public workshops, providing us with valuable information about example ontologies and perspectives on challenges and opportunities to advance them (in the order in which they presented to the committee): David Danks, University of California, San Diego (who later joined the committee); Russell Poldrack, Stanford University; Deborah McGuinness, Rensselaer Polytechnic Institute; Thomas Insel, Stanford University; Bruce Cuthbert, the National Institute of Mental Health; Benjamin Lahey, University of Chicago; Susan Michie, University College, London; Robert West, University College, London; Sandro Galea, Boston University School of Public Health; Howard Koh, Harvard T.H. Chan School of Public Health; Kathryn Phillips, University of California San Francisco; and Richard Moser and Lyubov Remennik, National Cancer Institute.

The committee also gratefully acknowledges the time that Anne Harrington, Harvard University, and Kenneth Kendler, Virginia Commonwealth University, spent answering our questions. We also thank Randall Engle, Georgia Institute of Technology, who had to resign from the committee very early in the project.

The committee commissioned five scholars to look more deeply at a number of questions. We thank the authors for producing excellent resources for the committee in record time: David Cella, Northwestern University, and Ronald Hays, University of California Los Angeles; Christopher Chute, Johns Hopkins University; Louise Falzon, the University of Sheffield; Janna Hastings, University College, London; and Kenneth Kendler, Virginia Commonwealth University.

I also want to express special gratitude to my fellow committee members who took time away from their busy schedules to delve into this complicated problem. This report is about shared conceptualizations and common scientific languages, and the committee itself was composed of experts from a wide range of academic backgrounds. Despite the fact that the project coincided with the COVID-19 pandemic and therefore had to be conducted almost entirely by Zoom, our work was characterized by a high degree of mutual respect and collegiality. Each member made significant contributions and each of us came away with a deeper appreciation for cross-disciplinary communication.

Finally, we owe very special thanks to Alexandra (Alix) Beatty, our exceptionally talented and experienced study director. Not only did Alix manage every decision and scrutinize every word in this report, she also ensured that the project remained on time. On multiple occasions the committee argued that it was not possible to produce the report on the scheduled time-line: Alix, with great poise and empathy, politely but firmly explained that taking more time was not an option. We were ultimately pleased that she

kept us on track while insisting on the highest level of quality. We also owe gratitude to Tina Winters for overseeing the technical aspects of constructing the report and many other challenging tasks and to Ashton Bullock for overseeing the administrative and logistical details.

This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.

We thank the following individuals for their review of this report: Michael Anderson, Brain and Mind Institute, University of Western Ontario; John Graybeal, Stanford University; Gizem Korkmaz, Biocomplexity Institute & Initiative, University of Virginia; Benjamin Lahey, Biological Sciences Division, The University of Chicago; Russell Poldrack, Department of Psychology, Stanford University; Jodi Schneider, School of Information Sciences, University of Illinois Urbana-Champaign; William Stead, Vanderbilt University; Frank van Harmelen, Computer Science Department, Vrije Universiteit Amsterdam; Timothy Wilson, Department of Psychology, University of Virginia.

Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by Cynthia M. Beall, Department of Anthropology, Case Western University, and Bernice A. Pescosolido, Department of Sociology, Indiana University. They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.

Coming together to produce this report has been an exceptional experience. When we began the process, many of us did not understand the depth of problems caused by mismatched conceptualizations, information overload, and lost opportunities to develop more coherent behavioral sciences. We leave the process with a better understanding of the issues and the promise that integrating the behavioral sciences with information and computer sciences will lead to the acceleration of knowledge.

This page intentionally left blank.

Contents

SUMMARY

1 INTRODUCTION

Context

A Long History of “Classification”

Particular Challenges in the Behavioral Sciences

Committee’s Approach to Its Charge

The Behavioral Sciences

Use Cases

Study Process: Four Key Questions

Guide to This Report

References

2 WHY ONTOLOGIES MATTER

Challenges with Synthesizing and Applying Knowledge

Challenges with Generalizing Research Findings

Challenges with Building and Structuring Knowledge

Challenges with Classification

Challenges with Defining Constructs

Challenges with Measuring Constructs

Summary

References

3 UNDERSTANDING ONTOLOGIES

Defining Ontology

A Continuum of Semantic Specification

Examples of Ontological Systems

A Formally Specified Ontology: The Behavioral Change Intervention Ontology

Classification Systems for Mental Health Problems

A Categorical Classification System: The Diagnostic and Statistical Manual of Mental Disorders

A Dimensional Classification System: Research Domain Criteria

A Quantitative Approach: The Hierarchical Taxonomy of Psychopathology

Conclusions

References

4 HOW ONTOLOGIES FACILITATE SCIENCE

How Ontologies Facilitate Scientific Progress

Clarifying the Phenomena That Are Studied

Classification

Communication

Data Integration

Data Sharing

Bibliographic Retrieval

Comparison and Analysis of Data

Primary Benefits of Ontologies

Improving Patient Care

Building Infrastructure for Scientific Research

Expanding Scientific Knowledge

Conclusion

References

5 ENGINEERING BEHAVIORAL ONTOLOGIES

Existing Behavioral Ontologies

The Ontology Engineering Process: Socio-Cognitive Practices

Creation

Change and Evolution

The Role of Institutions

Evaluation

The Ontology Engineering Process: Computational Tools

Creation and Editing

Dissemination

Evaluation and Debugging

Potential Directions for the Future

Needed Institutional and Organizational Support

Discovery

Capacity

Practices

Conclusions

References

6 CONCLUSIONS AND RECOMMENDATIONS

The Need for Ontologies in the Behavioral Sciences

Strengthening Ontology Use in the Behavioral Sciences

Supporting and Sustaining Behavioral Ontologies

References

APPENDIXES

A Ontological Systems Referenced in the Report

B Example Use Cases Generated in a Committee Self-Survey

C Biographical Sketches of Committee on Accelerating Behavioral Science through Ontology Development and Use

This page intentionally left blank.