4

Lessons Learned from Community Perspectives

The committee solicited community input via a call for white papers issued in December 2017 and presentations on targeted topics at two face-to-face meetings in November 2017 and January 2018. Appendix C includes the text of the white paper solicitation, a list of the white papers received, and a list of presentations made to the committee.¹ The committee strongly values the community input, and all papers were read and considered.

The white papers and the oral presentations articulated numerous areas of impact (both positive experiences and concerns) of a potential new open source software (OSS) policy for NASA’s Science Mission Directorate (SMD). The input was aggregated and organized here into big-picture topics, more-focused implications, and procedural areas of impact.

Overall, the community input expressed broad support for OSS. Openness and transparency are seen as central to scientific validity and reproducibility, but various challenges appear in the implementation of policy. A majority expressed positive experiences in opening code, describing a range of advantages, including efficiency, greater collaboration, and more robust code. Many white papers, however, emphasized issues and even pitfalls when trying to regulate the open sourcing of software. Concerns included legal ramifications, institutional barriers, costs, and the impact on individual scientists and their careers. Some suggested that an open source policy may not always benefit science, because for researchers, time spent publishing software comes at the expense of time spent doing science.² While an open source policy may enhance science for other researchers, it could be at the expense of the original researcher’s scientific output. In addition, there are concerns that researchers may lose motivation to push the boundaries of innovation in their software if they know that they have to immediately release it to the general public instead of having several years to take advantage of the new technology, potentially leading to less innovation in software development. Since doing science and developing OSS are different but complementary activities with different motivations and outcomes, OSS policies may be more successful if they clearly identify value in both activities. Many concerns reflect misunderstandings about open source licensing and processes. Others reveal legitimate legal and institutional barriers. Most unease stems from the culture of how science is currently competed and conducted. While supporting the principle of open dissemination of federally funded research, several white papers emphasized that substantial support is required to strengthen NASA computational capabilities and to build and sustain a successful OSS program.³ Some also find the comparison between open data and OSS to be mislead-

___________________

¹ White papers are referred to by number, and titles and authors are listed in Appendix C.

² White papers (WPs) 21, 29, 42, and 44.

³ See, for example, WP 22.

ing and suggest that software is more analogous to instruments.⁴ NASA SMD will need to address these concerns as it develops policies, but perhaps more importantly, it needs to foster a new culture of openness and encourage a social norm of sharing and collaboration. Work toward a cultural norm of openness has already begun with open data policies, support, and infrastructure. It needs to continue with carefully constructed support for OSS, beyond simple policy development and implementation. To achieve the full benefits of OSS, it is important to consider how a policy will interact with community norms.

Finding: The NASA science community generally recognizes the value of open source software and supports the principles of openness, but concerns prevail on the details of implementation and the impact on science and scientific careers.

Recommendation: NASA Science Mission Directorate should explicitly recognize the scientific value of open source software and incentivize its development and support, with the goal that open source science software becomes routine scientific practice.

This chapter addresses the perceptions of the community and the current state of OSS and provides some conclusions and recommendations for NASA SMD to remove barriers and foster the move toward openness, regardless of the choice of future policy.

4.1 IMPACT OF OPEN CODE

4.1.1 Software Reuse

Software reuse is a valuable result of open software. Well-written and well-documented community-developed OSS that is reused by many researchers reduces not only labor costs but also the chance of unintended errors (e.g., AstroPy, GDAL, scikit-learn; see WP 24). Instead of writing all analysis software oneself, using a community library can reduce the time spend coding, allowing the focus to remain on designing the analysis and interpreting results. An open code policy can also enhance open data policies, since data that is technically “open” can be effectively locked away for broad-based studies if the source code needed to read the data is not also open (WP 40). Open source can also improve the longevity of software. Well-documented and open software remains available and usable even if the original authors change institutions or retire. This ensures that data from experiments that may have cost millions of dollars in public funds can continue to be accessed and are not lost to future use due to closed software.⁵ OSS can protect individual researchers by ensuring that they can always work with their creations even when they change institutions. Additional users may discover new things about their code, including bugs and new unanticipated applications.

Creating high-quality software requires more effort than simple single-use software, whether open or closed. Code needs to be well documented and capable of responding gracefully to inadequately specified or incorrect arguments. The costs associated with development of an OSS policy will be discussed further in Section 4.3.

While some white papers, as discussed above, were positive about software reuse, some community members expressed reluctance to making software open source because of concerns about misuse.

1. A user could apply a piece of software in a regime where it is invalid or may not recognize a numerical issue in a simulation result.⁶
2. The reputation of the original software developer might suffer due to errors made later by others.⁷
3. The original developer may have to spend time ensuring that software is used correctly.⁸

___________________

⁴ See, for example, WP 29 and 31.

⁵ WP 40.

⁶ WP 21.

⁷ WP 35.

⁸ WP 6 and 22.

Following software development best practices and peer review of publications can alleviate some of these concerns. For example, publishing software on a repository with a citable digital object identifier (DOI) creates a permanent version of the software, while others can modify the source code by adding a “branch” on a version-controlled repository. The primary developer can maintain control to any changes made to the primary source code so that if an error were introduced, the responsible party is identifiable through the version-control features.

Finding: Open source software enables community members to build on each other’s labor and reduces duplication of effort. Especially for software that is broadly applicable, open source development can be more efficient and productive.

Finding: As more software is made open source, scientists will change how they work and how they evaluate each other’s work, yet there is anxiety in the community about this impending change.

4.1.2 Collaboration and Inclusion

Some researchers report that open sourcing their software allowed them to broaden their collaborative network, advancing code capabilities in unexpected ways and improving their citation counts (Figure 4.1). All these factors enhance the software’s impact on science (WP 7, 44). Others believe that OSS led to a broader collaboration network for the developer (WP 7).

Finding: Open source software can lead to a broader user and developer base for software, often fostering collaborations that are beyond what the original developers envisioned.

4.1.3 When to Open Software

Researchers sometimes find it difficult to determine exactly when to transition software to open source. Many science codes develop in stages, beginning as a research code that develops into a production code. For example, single-use software may transition to analysis software, or to simulation and model software. Scientists initially develop a research code to solve a specific problem or series of problems. On a time frame that can be much longer than a typical grant-funding period, scientists continue to develop and test the software, applying it to an increasing number of problems. During this period, the scientist may gain a set of friendly users (by-request access) who further test the software on a wider set of applications. This allows the developer to both fully test the software and also have time to publish a series of papers (return on investment) prior to releasing. Open sourcing the software occurs after this, sometimes lengthy, development/testing phase. Some express concern that transitioning to an open source requirement within a grant period may not allow developers the time to fully test the code or complete their own scientific findings with the code before releasing it to the broader community.⁹

In other cases, especially for software focusing on data mining and management (e.g., AstroPy), open sourcing the code from inception has produced not only a broad user group, but also a broad developer team. The broad developer base avoids duplication of effort and improves interoperability.

Conclusion: For many software projects, open sourcing the code from inception is ideal. For others, a period to verify and validate the code in a research mode may be a better approach.

4.1.4 Transparency and Reproducibility

Strong drivers for open data and OSS are the principles of reproducibility and replicability. Publications with only natural-language descriptions of methods, algorithms, and code implementation may be insufficiently

___________________

⁹ M. Kunz, 2018, “My (Biased) Take On: Experiences and Challenges in Open Source Policies,” presentation to the committee on January 17, 2018.

reproducible and are sometimes misleading. Ince et al. (2012)¹⁰ offer two examples. In the first example, data of global temperature anomaly released by the U.K. Meteorological Office included some errors that could have been discovered earlier if the software that processed the data had been open source. In the second example, a comparison of nine implementations of the same seismic data-processing algorithms revealed differences attributable to software errors. Ince et al. assert that ambiguity in descriptions using natural language are unavoidable, citing several works from software-engineering research literature. Even if description ambiguity could be eliminated, software errors are unavoidable. Some estimates report that 1 to 10 errors per thousand lines of code are typical.¹¹

Strict reproducibility (the ability to reproduce results with the same data and code) requires OSS and open access to all the needed metadata including initial conditions, data inputs, libraries, compiling requirements,

___________________

¹⁰ D.C. Ince, L. Hatton, and J. Graham-Cumming, 2012, The case for open computer programs, 482(7286):485.

¹¹ B. Boehm, H.D. Rombach, and M.V. Zelkowitz (eds.), 2005, Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili, Springer, New York.

computing environments, and so on. See, for example, Donoho et al. (2009)¹² and Nosek et al. (2015).¹³ This can be difficult to achieve, especially in high-performance computing applications that require large portions of supercomputer facilities. It further requires careful bookkeeping to ensure that everything that is needed to produce the exact published result is archived. Reproducibility of complex computational environments can be facilitated with new technologies, such as containers. Tools such as Docker, Vagrant, and VirtualBox help researchers re-create a computational environment from a list of requirements. These new technologies are making reproducibility easier and are reducing barriers for building on past work. Yet, being able to reproduce a result with a specified metadata and a given code does not ensure that the results or code are correct. Replication requires that researchers use independent methods to achieve consistent results, verifying the scientific findings. Many communities already have a strong replication tradition, where trust in any scientific result is built when multiple codes achieve results that demonstrate consistent behaviors. By requiring multiple codes to achieve the same scientific finding, replication reduces the impact of individual code errors or numerical issues.

At least one journal, the American Journal of Political Science (AJPS), has adopted a rigorous reproducibility standard, including verification of each compendium by an independent organization. It is a costly process (approximately $1,000 per article), and fewer than 300 papers are published each year, with a 90 percent rejection rate, but AJPS’s reputation has only increased since a adopting the policy, a sign of readers’ trust in independently verified reproducible results.¹⁴ While this type of compliance checking is not easily scalable to thousands of papers per year, journals are moving in this direction.

Reproducibility requires full transparency, enabling scientists to better independently replicate results. It is often difficult to fully describe in a publication the methods that were used with sufficient enough detail to reproduce the results. This is rarely intentional, but often details are missed in the capturing of the method in a publication. By opening software, scientists can more readily do code/method comparison, allowing for replication. OSS facilitates broader code review and builds trust among scientists and code projects.

Finding: Reproducible research requires both open source software and good metadata, including initial conditions, data inputs, libraries, compiling requirements, computing environments, and so on.

Finding: For many research projects, reproducing all results is neither tractable nor does it ensure that the results are correct. Replicability is also important. However, the transparency provided from open source software can foster and improve both replicability as well as reproducibility.

The committee found through presentations and discussions that the community has concerns regarding how OSS policies could potentially be exploited to “scoop” results. Some efforts have been made through licensing to mitigate concerns about first publication of results, but these licenses do not adhere to currently popular opensource definitions. Additionally, the publication of OSS would ensure that appropriate credit is given by providing a traceable record of the software publication. Making reproducible research a community norm will more likely be accomplished through incentives such as badging and journal or funder policies. NASA SMD may want to support that norm while ensuring fair credit and reasonable protection for researchers.

4.1.5 Institutional Challenges

OSS requires a change not only in the attitudes of scientists, but also in institutions. Some universities and companies tend to protect their intellectual property and may be reluctant to support OSS or may use restrictive licenses that could be in conflict with a future NASA policy. Some institutions, often for legitimate legal reasons, have developed a cumbersome system to release software as open source. Including an OSS condition at the

___________________

¹² D. Donoho et al., 2009, Reproducible research in computational harmonic analysis, Computer Science and Engineering 11(1):8-18, doi: 10.1109/MCSE.2009.15.

¹³ B.A. Nosek et al., 2015, Promoting an open research culture, Science 348(6242):1422-1425.

¹⁴ W.G. Jacoby, 2018, “The Replication and Verification Policy at the American Journal of Political Science,” presentation to the committee on January 18, 2018.

grant proposal stage could encourage institutions to streamline, where possible, their approval processes, so as to meet funded contract or grant obligations. Institutions that receive grants from NASA are required to return annual reports on research progress, which may include progress on data and software development and release. The program manager overseeing the grant authorizes each year’s funding increment. If a report was unable to show the release of software in a timely manner, this could affect the annual funding authorization or proposals for future grants, which may ask for documented sharing practices.

NASA’s internal processes can also be cumbersome. One civil servant described how it took her “five months and 38 pages of paperwork to release 217 lines of non-sensitive code” (WP 1). The NASA Technology Transfer Program has worked hard to streamline this process,¹⁵ and some NASA researchers have noted the improvements. At the same time, the chief information officer (CIO) for NASA Headquarters is encouraging a general approach.¹⁶ The code.nasa.gov website outlines steps for scientists at NASA centers to follow in releasing software and already provides access to hundreds of software packages. It states, “Depending on the number of projects being assessed for release at any given time general workloads and backlogs, traversing the release process can take anywhere from 3 to 6 months.”¹⁷ Greater coordination between the CIO and Technology Transfer Office could clarify and accelerate the open source process. Since NASA employees also compete for funding, a NASA policy to release software as open source could encourage streamlined software release policies and procedures.

NASA’s relations with contractors and grantees present other issues. The language used in NASA Federal Acquisition Regulations (FAR) conflates copyright ownership and the ability to distribute copyrighted materials, bundles software into data, and explicitly discourages open source (WP 12).

Conclusion: Achieving the goal of establishing a social norm of open source software requires altering the views of institutional management in addition to scientists.

4.2 EDUCATION AND TRAINING NEEDS

Education underpins all efforts to move the NASA science community toward acceptance of OSS. Software development experience varies widely across and within scientific disciplines, and the education component of any policy implementation needs to account for these differences. For example, in a 2015 informal survey of 1,142 astronomers, “only 8% of them report that they have received substantial training in software development. Another 49% of the participants have received ‘little’ training. The remaining 43% have received no training.”¹⁸ For example, some programs (e.g., ACCESS¹⁹) have projects led by professional software developers, but most SMD projects are led by scientists, who may not be as conversant with the latest coding standards or techniques. A move toward OSS is seen by many researchers as imposing extra requirements unrelated to or detrimental to the scientific quality of a project (WP 6, 9, 10, 15, 21, 22, 26). Clear communication with the research community about new requirements is essential for success. Training scientists in best software practices is critical to gaining the full benefit of OSS (WP37). Convincing them of the long-term benefits to their science is the best (perhaps only) way to gain their acceptance of an OSS policy. There are already strong community efforts to educate scientists about software development. Rather than replicating them, SMD could sponsor events such as face-to-face Software Carpentry²⁰ workshops, online classes,²¹ or mentorship workshops for late-career scientists. For example, Software Carpentry workshops do not tell participants how to write their code, but they teach them how to use version control software and how to create a citable persistent identifier (e.g., DOI) for their software using Zenodo.²²Figure 4.2 outlines

___________________

¹⁵ D. Lockney, 2018, “NASA Software Release,” presentation to the committee on January 18, 2018.

¹⁶ Jason Duley, discussion with the committee, February 16, 2018.

¹⁷ See https://code.nasa.gov/#/SRA.

¹⁸ I. Momcheva and E. Tollerud, 2015, “Software Use in Astronomy: An Informal Survey,” arXiv:1507.03989v1.

¹⁹ Advancing Collaborative Connections for Earth System Science (ACCESS), https://earthdata.nasa.gov/community/community-data-systemprograms/access-projects.

²⁰ See http://software-carpentry.org, a nonprofit that focuses on teaching people to code.

²¹ For example, https://codecademy.com/ and https://datacamp.com/, for profit companies that have online classes teaching coding.

²² Zenodo is an online repository for research output. Zenodo was created by Open Access Infrastructure for Research in Europe (OpenAIRE) and CERN and is supported by the European Commission, https://zenodo.org/.

the basic process of how a software developer could take software and make it open source using both GitHub and Zenodo. While there are many steps, they are all relatively straightforward, with the most complicated one being choosing a license. Once this is done, publishing the code and getting a DOI that can be used in publications is simple. Additional topics for education include but are not limited to the following: (1) how to organize and publish software in a repository (e.g., GitHub); (2) how to choose and include a license; (3) what to consider before sharing software (university rules, export controls, etc.); and (4) other OSS best practices.

Finding: Community understanding, experience, and familiarity with open source software is essential to the acceptance of open source software as a tool to enhance scientific research.

Two community deficiencies were of particular concern: the lack of training in software development and scientific computing and the missing guidance on legal issues for scientists.

4.2.1 Modern Computing

Some committee members noted, from their own experience, that young scientists may not have adequate training in modern computing and software development and are likely to reuse existing code rather than create new code. This presents both advantages and disadvantages. Reuse can help a beginning programmer learn while creating software faster and more efficiently, but using these programming tools could circumvent understanding what a piece of code actually does. Science is increasingly dependent on software for analysis, but many scientists lack the training in software development best practices and this is a barrier to collaborations. This is a national trend that is likely due to a multitude of factors. Regardless of the cause, communication of this skill mismatch, and the fact that coding is becoming as essential as calculus to scientists, could motivate secondary schools and colleges to include software development best practices in their curricula for all science, technology, engineering, and mathematics (STEM)-bound students. OSS provides a way to educate and train new talent (WP 40).

4.2.2 Guidance on Legal Issues

Determining whether something is export controlled is nontrivial, but if an OSS policy is to succeed, it will have to become trivial (WP 5). Much of the Earth and space science community lacks basic knowledge about intellectual property law, contractual wording, software licensing, institutional policies, and export control, because this knowledge has not been required for their research. While most research is not export controlled, checking for sensitive technologies is part of due diligence when releasing software. In some cases, safely releasing software is simply a matter of finding the correct legal resources within a scientist’s institution. For smaller institutions or independent scientists, guidance may be hard to find. Implementing a policy without ensuring the necessary access to legal resources for NASA-funded scientists would impose a burden (WP 8) and create inequalities in access to legal advice (WP 15).

NASA has several sites for releasing software that begin to address some of these concerns, but they are not well known. The code.nasa.gov website gives a step-by-step procedure for NASA employees to release OSS (Section 3.1.1). At software.nasa.gov, any user can request software, but the release type enforces export controls. WP 5 documented the difficulty one NASA employee experienced while trying to understand whether the software he wanted to release was export controlled. Better informing the community of such sites, and further developing them, could provide a centralized distribution and education facility for SMD-funded OSS.

Finding: The scientific community’s lack of knowledge regarding the legal issues is a currently a barrier to releasing open source software.

Conclusion: Training and education on legal issues and open source software best practices will improve acceptance of any new requirements.

Recommendation: NASA Science Mission Directorate should initiate and sponsor programs to educate and train researchers in open source best practices. Topics could include, but are not limited to, export controls, licensing and intellectual property, workflows, and software development. These resources could be made available to the community via in-person trainings as well as webpages, screencasts, and webinars.

4.3 FUNDING AND EFFORT NEEDS

It is important that NASA consider the cost of a future OSS policy against the impact on innovation and scientific productivity and act to minimize new non-science-related burdens for science proposals, including training requirements when implementing any OSS policies.

4.3.1 Funding

Despite many community members expressing support of OSS because it has advanced their research and careers, as discussed above, others are concerned that releasing OSS may disadvantage those scientists who have spent time on the code development (WP 15, 21, 26, 29, 32). Because funding and career advancement are tied to traditional scientific output (e.g., publications) rather than code development, some researchers feel the need to maintain closed software in order to have an advantage when securing funding. Similar concerns were expressed when NASA adopted an open data policy.

There is also a concern that NASA’s current funding mechanisms do not support the time it takes to document, test, and maintain robust, reproducible, and reusable software. Although open source does not require these elements, they are necessary to maximize its benefits, and sometimes are implicitly expected (WP 11, 15, 37). Many believe that making the analysis code available as is (i.e., without documentation) is unlikely to enhance scientific productivity (WP 16 and 21).

Another concern of the community is that any mandated OSS policy without associated institutional and financial support would be untenable to scientific code developers (WP 6, 7, 12, 15, 18, 19, 24, 31, 41, 44). If there are increased costs to software development imposed by NASA without clear funding channels, developers may choose to work on non-NASA projects with funding from other sources. Additional funds to open source codes will help facilitate more OSS. The cost may be prohibitive for legacy codes (WP 32). Maintaining open source libraries and software requires the allocation of resources (WP 7, 8, 10, 11, 22, 33, 39, 44).

Finding: Making code open source is valuable, but NASA Science Mission Directorate will need to consider the additional costs to the developers and to NASA. For many codes, additional funding will be needed to document, test, and maintain robust, reproducible, and reusable software. While the benefits of open source software start at the simplest step of openly sharing software, to achieve the full benefits of open source software, adequate funding will be required.

4.3.2 Effort

OSS development enables software reuse, avoiding duplication of effort (WP 3, 26). Open code also encourages a larger developer base, which can contribute to maintenance, verification, and validation of the code (WP 3, 16, 34, 39).

OSS does require time and effort by both researchers and institutions to identify the appropriate license, implement adequate development practices, and so on. Finding and implementing all the open source components used by software can be difficult and time consuming, complicating future projects (WP 10, 13, 43). An example complication is when software projects are funded by multiple agencies with different open software policies. Another example is that the use of commercial-off-the-shelf (COTS) software in an era of open source may cause confusion unless the policy is very clear. There is concern that new policies may preclude the use of COTS tools to support research, which could have a negative impact (WP 13).

In addition, many scientists feel compelled to “clean up” and document code before open sourcing (WP 6, 10, 18, 40). Once software is open, additional work is required to provide support and enhance an existing code. Although this could lead to higher-quality codes, there is some concern that scientific productivity and innovation could be negatively impacted by this extra work.

Making single-use code open source may not be worth the extra effort (WP 6, 13, 18, 21). Single-use codes can evolve into production codes with broader applications, at which point benefits from open source development could be larger and, for some software, delaying the time to open source may be warranted (WP 29).^23,24

Finding: The perception that considerable effort is required for open source software may inhibit its adoption.

___________________

²³ P. Woodward, 2018, “Thoughts on Open Code Policy,” presentation to the committee on January 17, 2018.

²⁴ M. Kunz, 2018, “My (Biased) Take On: Experiences and Challenges in Open Source Policies,” presentation to the committee on January 17, 2018.

Conclusion: An incremental approach to open source software will allow researchers to adjust to new requirements and minimize the impact on their scientific productivity.

Conclusion: Flexibility is needed in an open source policy.

Recommendation: Any open source software policy that NASA Science Mission Directorate develops should not impose an undue burden on researchers; therefore, any policy should be as simple as possible, and any mandates should be fully funded.

4.3.3 Support for Good Practice, Governance, Maintenance, and Infrastructure

The implementation of a NASA OSS policy necessitates a governance infrastructure to ensure software quality and security. This type of infrastructure requires a sustained financial investment with clear roles and responsibilities for staffing and participation (WP 37). Lessons learned from the implementation of NASA’s open data policy illustrate the importance of design and planning of a supporting infrastructure for open software prior to any policy implementation.

As research scientists publish their software and use community resources, such as tools and libraries, the software needs to be managed, discoverable, and, where appropriate, maintained in a centralized (as much as possible) repository (WP 30). There are existing NASA data resources that publish sensitive software²⁵ and general open software²⁶ that could be leveraged for new OSS initiatives. In addition, NASA can encourage the open source development model described in Section 2.5.

Recommendation: NASA Science Mission Directorate should support the infrastructure, governance, and maintenance of a healthy open source community, taking advantage of existing community resources to the greatest extent possible.

4.3.4 Support for Community Software

NASA science is enabled and dependent on open source, community-developed software, from fundamental packages such as NumPy, which performs standard array operations, to domain-specific ones such as AstroPy, which enables many common astronomical calculations. The continued existence, active development, and widespread adoption of open source libraries by many federally funded projects demonstrate the power of community-developed software. Some successful open source projects are sustained by organizations (often private companies) that gain training for future employees and libraries for existing employees to use, and community goodwill. However, most open source libraries are currently maintained by volunteers and lack clear sustainability plans. This lack of institutional support, dedicated full-time developers, and dedicated funding for these libraries represents a major vulnerability of the basic infrastructure of NASA science. Muna et al. (2016)²⁷ described several options for securing this infrastructure. NASA could allocate professional software developers to these projects, could encourage NASA employees to participate in the projects, or could provide funding directly to the projects to hire the needed developers. An example of a government agency supporting open source libraries and working with the existing open source community is the Department of Defense (DOD) Defense Advanced Research Projects Agency (DARPA) Open Catalog.²⁸

Finding: NASA science already depends on open source, community-developed libraries.

___________________

²⁵ See https://software.nasa.gov.

²⁶ See https://code.nasa.gov.

²⁷ See https://arxiv.org/abs/1610.03159.

²⁸ See https://opencatalog.darpa.mil.

Finding: Science built on open source software is most effective when it has a strong, coordinated, and active community.

Conclusion: Community software provides substantial value to SMD-funded researchers, and NASA’s recognition and support of these software projects is vital.

Recommendation: NASA Science Mission Directorate should support open source community-developed libraries that advance NASA science.

4.4 ENABLING CREDIT AND CAREER ADVANCEMENT

Community members are concerned that making software open source makes their intellectual property vulnerable to reuse without attribution (WP 6, 15, 20, 21, 42).^29,30 In the past, developers of OSS have struggled to obtain permanent positions in science (WP 44). Although new policies are being developed to ensure attribution (WP 7, 44) for these efforts, more work is possible to protect the careers of open source developers.

Moving to OSS can help with career advancement, especially outside academia. Examples exist where software is held by an institution, and, when a scientist leaves, this software stays at the institution, often with no viable support mechanism. OSS would allow the scientist to continue building a career using the software. Open software protects scientists from companies, universities, or laboratories interfering with their work when they move between institutions. But despite recent improvements, the perception remains of a lack of respect and appreciation in academia for software development. NASA investments can help change this perception. Recognizing science software as a critical element for innovation and investing through a long-term software development and maintenance program may elevate the status of the software developer. Academic credit for OSS is important as well, especially through formal citation in the scientific literature and encouraging the acceptance of OSS as a community norm. As described in Chapter 3, some journals are beginning to require that the software used to create a research result be made available. Ideally, this would facilitate formal citation using a persistent identifier such as a DOI. Software citation can promote both scientific reuse and formal recognition of software developers, but software citation practices are not yet firmly established. Several journals, including SoftwareX, the Journal of Open Research Software, and the Journal of Open Source Software, provide a means to publish peer-reviewed “software papers” that include release of the described software. This is appropriate for some but not all software; yet, much like data, any software used to produce a published scientific result needs to be cited. Data citation principles and practices are becoming more firmly established,^31,32 and similar principles for software citation are developing, but practices are not as mature.³³ Furthermore, citation is only one method of addressing credit. Projects such as depsy,³⁴ Project CRediT,³⁵ and others have explored new ways to understand different types of contributions and the general concept of “transitive credit.” Note that changes or adaptations to software by someone other than the original author are more easily done on software openly released on GitHub or similar repositories, where the original software is clearly and persistently identified, and each individual’s contributions or separate work are identified through “commits.”

NASA can encourage the further development of appropriate credit schemes by formally recognizing software contributions in grant and contract proposals, interim and final project reports, and hiring and performance

___________________

²⁹ M. Kunz, 2018, “My (Biased) Take On: Experiences and Challenges in Open Source Policies,” presentation to the committee on January 17, 2018.

³⁰ W.R. Hix, 2018, “One Programmer’s Experience and Challenges on Open Source Policy Decisions,” presentation to the committee on January 17, 2018.

³¹ Data Citation Synthesis Group, 2014, Joint Declaration of Data Citation Principles, M. Martone (ed.), Force11, San Diego, https://doi.org/10.25490/a97f-egyk.

³² J. Starr, E. Castro, M. Crosas, M. Dumontier, R.R. Downs, R. Duerr, L.L. Haak, M. Haendel, I. Herman, and S. Hodson, 2015, Achieving human and machine accessibility of cited data in scholarly publications, Peer Journal Computer Science 1:e1-, http://dx.doi.org/10.7717/peerj-cs.1.

³³ A.M. Smith, D.S. Katz, K.E. Niemeyer, and FORCE11 Software Citation Working Group, 2016, Software citation principles, Peer Journal Computer Science 2:e86, https://doi.org/10.7717/peerj-cs.86.

³⁴Nature article describing depsy.org at https://www.nature.com/news/the-unsung-heroes-of-scientific-software-1.19100.

³⁵ See http://casrai.org/credit.

review processes. The goal is to foster the norm of OSS by explicitly documenting and registering contributions and recognizing the value of those who contribute.

Conclusion: In order to recruit, retain, and support the skills base, software development efforts need to be rewarded with academic credit (e.g., grants, publications, citations, fellowships, and prizes).

Recommendation: NASA Science Mission Directorate should foster career credit for scientific software development by encouraging publications, citations, and other recognition of software created as part of NASA-funded research.