2

Program Management

With the fourth largest federal Small Business Innovation Research (SBIR) program in terms of SBIR dollar obligations, the National Aeronautics and Space Administration (NASA) obligated $141.8 million in SBIR awards in FY2014. During the (FY) 2005-2014 period, NASA SBIR funds supported an annual average of 318 Phase I and 135 Phase II awards, all of which were contracts rather than grants.

This chapter reviews key features of the NASA SBIR program and highlights issues and concerns about its management. More recent initiatives within the program are discussed separately in Chapter 3. Sources for this chapter include discussions with NASA staff, information from the 2011 Survey¹ and company case studies, and documentation from NASA.

NASA PROGRAM ORGANIZATION

As a part of the agency’s reorganization, NASA’s SBIR program is now located within the Space Technology Mission Directorate. The Program Executive at NASA Headquarters provides strategy and guidance, but operations, including decisions about awards (see below), are handled within the Field Centers and Mission Directorates. The NASA organization chart showing the SBIR program is provided in Figure 2-1.

__________________

COMPANY ELIGIBILITY

As described in the Small Business Administration (SBA) Policy Directive that governs program administration, an applicant to the SBIR program must be a for-profit business located in the United States and must be more than 50 percent directly owned and controlled by one or more individuals who are citizens or permanent resident aliens of the United States, by other business concerns each of which is more than 50 percent directly owned and controlled by individuals who are citizens or permanent resident aliens of the United States, or any combination of these.² Up to 15 percent of NASA SBIR funds may be awarded to small businesses that are majority owned by multiple venture capital, hedge fund, or private equity firms.³ The principal investigator must be employed by the small business at least two-thirds time for Phase I and half time for Phase II.

__________________

In addition, the 2011 SBIR reauthorization act required that firms winning several Phase I awards should provide evidence of successful transition to Phase II. Writing in the NASA SBIR newsletter, Rich Leshner, a former program director, stated, “To be eligible for a Phase I award, proposers that have previously won over 20 Phase I SBIR/STTR awards over the past 5 years (excluding the most recently completed fiscal year) must satisfy the Phase I-Phase II transition rate benchmark, which requires SBIR/STTR firms to have at least a 25 percent Phase I to Phase II success rate.”⁴ However, NASA has not yet excluded firms on this basis, perhaps reflecting the fact that this 25 percent benchmark is well below the average transition rate of 42 percent.

TOPICS AND APPLICATIONS

Consistent with the 2011 reauthorization act, NASA has increased the size of both Phase I and Phase II awards. Phase I now provides maximum funding of $125,000, with a maximum duration of 6 months for SBIR and 12 months for Small Business Technology Transfer (STTR) awards. Phase II awards are for 24 months, with a maximum award of $750,000. NASA guidelines expect that Phase II will result in “the delivery of a prototype unit or software package, or a more complete product or service, for NASA testing and utilization.”⁵ In addition to the standard Phase I and Phase II programs, NASA offers two additional funding mechanisms, Phase II-Enhancement (Phase II-E) and Phase II-EXpanded (Phase II-X).⁶ These program enhancements, summarized in Table 2-1, are discussed in detail in Chapter 3.

Single Solicitation

NASA continues to offer one application period per year. Other agencies have moved toward multiple annual solicitations in recognition of the accelerating pace of technical change. Because the NASA SBIR program offers tightly defined topics that may not be well suited to some potential applicants, this limited window has a significant impact. Small, often fragile companies may not be able to wait 12 months for the next opportunity to apply.

Topic Selection

As a part of a 2007 reorganization of NASA, the SBIR program became the responsibility of the four NASA Mission Directorates. The program office is now located within the Space Technology Mission Directorate,⁷ and

__________________

TABLE 2-1 Funding and Duration for NASA SBIR, STTR, and SBIR Select, by Phase

NOTE: SBIR Select is a new initiative from NASA, starting in 2013, which provides up to $1.5 million in Phase II awards for companies whose technologies are of more immediate interest to NASA.

SOURCE: NASA, The Concept, 4(1), Winter 2013.

each Mission Directorate provides a representative to the SBIR program, each of whom is responsible for ensuring that SBIR topics meet the mission needs of the particular Mission Directorate. Subtopics can be nominated by anyone at NASA, provided they align with broader topics defined by the Mission Directorate representatives. Individual NASA centers do not set quotas for SBIR subtopics.⁸ (See Figure 2-2.)

Once the subtopic nomination period closes, each Mission Directorate reviews its subtopics. Decisions on which subtopics to approve are usually made by committees of senior technical staff within the Mission Directorates, although each Mission Directorate can make these decisions its own way. At this point in the process, the staff of each Mission Directorate can

__________________

identify subtopics of particular importance to its Mission Directorate, where successful technologies would be prime candidates for take-up (“infusion”) by the Mission Directorates. These subtopics can be nominated as “select subtopics.” This is a new initiative, starting in 2013, which provides up to $1.5 million in Phase II awards for companies whose technologies are of more immediate interest to NASA.⁹ According to Rich Leshner, former program director, these topics are focused on the following:

1. Projects with more immediate infusion opportunities (e.g., a Mission Directorate program has an immediate interest in using a successful prototype from Phase II);
2. Projects for which additional funding is justified to cross a boundary, which will, if successful, provide more than a 2:1 return; and
3. Projects that permit a science mission to be undertaken in a completely new way (e.g., a topic on balloon-based astronomy that was approved for “select” status at Marshall Space Center).¹⁰

__________________

Some NASA staff observed that this change in the topic development process has resulted in a tight focus on the short-term needs of Mission Directorates, often at the expense of more innovative or disruptive technologies. For example, NASA has, according to some staff, lagged in its efforts to introduce topics on information technology tools to help run its operations, which would in turn require a commitment to longer-term strategy.¹¹ Indeed, one staff member recommended that some funding should be set aside within the NASA program explicitly for more innovative projects. However, as Rich Leshner has observed, SBIR projects must be aligned with NASA Mission Directorate programs or they will not be adopted by NASA and the program will have limited value.¹²

Access to Program Staff During Solicitation Period

Once the NASA SBIR/STTR General Solicitation opens, NASA will only accept questions seeking clarification of proposal instructions and administrative matters. During this period, NASA staff is not permitted to answer questions about technical topics and/or subtopics. It should be noted, however, that it is not unusual for government solicitations to limit or control contact during a competition in order to preserve the fairness and appearance of fairness of the awards process.

Before the solicitation is published, NASA allows contacts from companies. NASA states that “Firms are encouraged to communicate with NASA mission program personnel and researchers to learn about the needs and objectives of mission programs.” The NASA SBIR/STTR website provides contact information for program contacts in the Program Management Office and in each of the NASA Centers for SBIR/STTR programs, by clicking on “Contact Us”. However, other than frequently asked questions (FAQ), there was no evidence that all questions and answers concerning solicitations are published online.

Somewhat different rules govern the new Select Solicitation: proposers can submit questions online via the NASA SBIR/STTR website for a period of 10 business days after the solicitation opens. This new approach suggests that NASA could also adopt a more open approach during the standard SBIR solicitation. Again, the questions and answers are not published online for all to see.

The arms-length approach adopted by NASA is likely to generate significant information gaps and difficulties for small companies. Given that questions are in fact permitted for select solicitations, the general prohibition outlined above is merely a matter of convenience for NASA; it saves technical

__________________

staff from the time and effort of having to answer questions, some of which will undoubtedly be irrelevant.

Application

The solicitation is open for 60 days. All applications must be made through NASA’s in-house Electronic Handbook (EHB), which provides guidance and eventually the electronic support for project management, tracking, and evaluation.¹³

Subtopic managers are responsible for identifying and selecting proposal reviewers, all of whom are from within NASA, because the agency does not use third-party reviewers. In general, experts are identified before the topic is formally published in the solicitation. A minimum of two expert reviewers are required; in some cases, three are used.

SELECTION

NASA uses a standard template for evaluating proposals. Scoring is broken out as follows:

• Technology: 50 percent
• Work plan: 25 percent
• Qualifications: 25 percent
• Community impact: textual response only

Proposals that score 85 percent or higher are judged to be “recommended.” According to agency staff, as much as 85 percent of Phase I applications meet this standard. The subtopic manager then ranks the recommended proposals (perhaps on the order of 50 proposals).

For Phase II only, proposers are required to include a commercialization plan, which is separately reviewed by contractors hired by NASA’s Jet Propulsion Laboratory. The contractors score each proposal on a five-point scale (excellent, very good, good, fair, and poor). However, these scores do not impact the numerical score of the proposal and do not affect the initial ranking.

The topic manager, assigned by the Mission Directorate, then collates and ranks all Phase II proposals for a given topic (which would include proposals from several subtopics). Phase II commercialization scores are primarily addressed at this point in the process. According to agency staff, this ranking is based more on agency mission needs, expressed as alignment with Mission Directorate objectives, than on the technical scores generated during initial review or the broader commercial potential of the project. As one staff

__________________

member noted, “In the end, we use a very simple metric—impact on NASA. That really drives selection, but we don’t really stress this even to companies.” An unpublished staff analysis indicated a very strong positive correlation between proposed impact on agency programs and selection.

Once the topic manager review is complete, the Mission Directorate convenes a board of all of its SBIR/STTR topic managers, which develops a final prioritized list of proposals. SBIR/STTR funds are distributed to the proposals that are identified as top priorities by the Mission Directorate boards of topic managers. Funding is proportionate to the overall NASA R&D budget. For example, aeronautics, which accounts for about 3 percent of NASA’s R&D budget, will receive about 3 percent of NASA SBIR/STTR funding.

Review procedures are apparently identical for Phase I and Phase II, with the exception of the commercialization review of Phase II awards discussed above. The selection process is entirely siloed within each Mission Directorate. No evidence was found of collaboration or communication across Mission Directorates in the selection of proposals.

Limits on Submissions and the Encouragement of New Participants

NASA currently imposes a limit of 10 proposals per company per solicitation.¹⁴ The limit at NASA, which was imposed in FY2007-2008, has made a difference: Several of the companies that have been the most successful in winning awards—such as Advanced Cooling Technologies, Creare, and Honeybee—have significantly reduced their number of applications. This limit may explain partly the about 20 percent decline in the number of proposals submitted between FY2007 and FY2008.

At least two rationales prevail for limiting the number of submissions at the individual company level. One is that a limit effectively pushes part of the review and decision-making process back onto the company, which best knows its own capabilities and interests. Another is that a limit enables increased attention to a greater variety of small companies.

CONTRACTING AND FUNDING

Winning proposals are announced on the NASA website. This announcement starts a process of contract negotiation between NASA contracting officers and the firm. This typically lasts about 2 months. At this point the contracts still require detailed cost justification. Contracting is handled by the NASA Shared Services Center (NSSC). One experienced NASA SBIR official noted that she had never seen a contract rejected at that stage.

__________________

Each contract requires that NASA be represented by a contracting officer representative (COR),¹⁵ who must undergo an initial 40-hour training and additional continuing education thereafter. Perhaps unsurprisingly, some NASA staff indicated that finding CORs for SBIR/STTR awards was sometimes a significant challenge. The training is primarily designed so that CORs can manage major contracts, and some NASA SBIR staff suggested that a less onerous training program (“COR Lite”) could be more appropriate for SBIR and also allow for rapid deployment of sufficiently qualified staff.

In FY2012, NASA shifted its base SBIR/STTR contract to become non-severable, that is, to provide for a commitment across the entire span of a 2-year Phase II award. As a result, there was no solicitation during 2013, because those funds were reallocated to fully fund the 2012 awards.

Phase I awards are reviewed in January/February and start in April/May. Phase II review starts in October/November, and Phase II awards are usually made in January of the following year, with contracts in place by February/March.

NASA does not permit no-cost extensions for the 6-month Phase I awards. Phase II applications are accepted during a short (2-week) time period at the end of the Phase I award. This timing suggests that the Phase II application process starts weeks before the end of the Phase I award. Given the very short timeframe of the NASA Phase I award, this suggests that projects would have an advantage if some Phase I technical results were known very early in the Phase I period, which in turn suggests that perhaps the program is not designed to support more ambitious efforts that might require longer time frames especially at the feasibility stage.

Funding Gaps

Funding gaps can develop between Phase I and Phase II of an SBIR award, creating challenges for small firms that are less likely to have other funding sources to sustain projects until Phase II funding arrives. Unlike other agencies, NASA does not offer bridge funding between Phase I and Phase II. It also does not offer a “work at your own risk” contracting approach. Under such a scheme, companies can proceed without a contract into Phase II, with costs reimbursed only if a Phase II contract is finalized.

More than 80 percent of NASA Phase II respondents to the 2011 Survey indicated they had experienced a gap between the end of Phase I and the start of Phase II for the surveyed award (see Table 2-2). A funding gap can have a range of consequences for a company, as presented in Table 2-3. Two-thirds of all respondents reported that they stopped work during this period, while a large

__________________

TABLE 2-2 Funding Gap Between NASA SBIR Phase I and Phase II Awards, Reported by 2011 Survey Respondents

NOTE: N=177 Respondents. According to NASA’s annual report to the SBA, in 2012, the lag between the end of Phase I and the beginning of Phase II averaged 233 days. The lag was 160 days between the date of notification of a Phase II award and the first day of performance under the contract. NASA Small Business Innovation Research (SBIR) Program Annual Report to the U.S. Small Business Administration for FY 2012, p. 8.

SOURCE: 2011 Survey, Question 22.

TABLE 2-3 NASA SBIR: Effects of Funding Gap on Surveyed Project, Reported by 2011 Survey Respondents

NOTE: N=146 Respondents.

SOURCE: 2011 Survey, Question 23.

majority of the remaining one-third worked at a reduced level of effort. One percent ceased operations. Aside from delaying projects, funding gaps can result in significant negative long-term consequences, especially for smaller companies, where in some cases there is insufficient work to retain key project staff during the gap period.

In some cases, the flow of funding from the agency to the awardee can be interrupted between phases of an SBIR award. According to NASA’s annual report to the SBA, in 2012, the lag between the end of Phase I and the beginning of Phase II averaged 233 days. The lag was 160 days between the date of notification of a Phase II award and the first day of performance under the contract.¹⁶ These lags present a major challenge to small firms that do not have other projects on which to place staff during the gap period.

__________________

Adequacy of Funding and Size of Awards

Although award recipients could be expected to say that the amount of money provided was not sufficient for a given project, there can be some value in determining the extent of alternative responses, particularly when the issue is posed as a trade-off between the number and the size of awards.

As reported in Table 2-4, 55 percent of 2011 Survey respondents indicated that the funding was sufficient, and 45 percent indicated that more funding was required. None reported that the funding was more than necessary. In other contexts (e.g., case studies), awardees often suggest that the size of the SBIR awards should be increased (a view especially prevalent before the recent changes in the 2011 SBIR reauthorization act). The 2011 Survey asked directly about the possible trade-off between the size of awards and the number of awards: unless agency funding for SBIR programs increases, larger awards inevitably imply fewer awards. In the context of that trade-off, there was no clear majority for (or against) an increase in the size of individual SBIR awards, although a plurality was opposed, as summarized in Table 2-5.

WORKING WITH THE CONTRACTING OFFICER TECHNICAL REPRESENTATIVE

Case studies of SBIR companies suggest that a critical factor affecting the success of SBIR projects is the relationship between the awardee and the

TABLE 2-4 NASA SBIR: Adequacy of Phase II Funding Reported by 2011 Survey Respondents

NOTE: N=179 Respondents.

SOURCE: 2011 Survey, Question 42.

TABLE 2-5 NASA SBIR: Respondent Views on Trade-off of Larger Awards for Fewer Awards (2011 Survey)

NOTE: N=179 Respondents.

SOURCE: 2011 Survey, Question 43.

agency’s project manager. At NASA, the latter is called the contracting officer technical representative (COTR). The 2011 Survey asked a series of questions aimed at identifying ways in which this relationship might be improved.

We hypothesized that, in the absence of a NASA-wide standard, there might be wide variation in the degree to which COTRs actually engage with their awardee projects. When asked how often they engaged with their COTR, just under one-half of respondents reported monthly contact, while 38 percent reported quarterly contact (see Table 2-6).

Case studies of SBIR companies revealed that some COTRs had very positive effects on their awardee companies, while others were of little help. The 2011 Survey attempted to gauge the distribution of utility by asking respondents about their COTR’s usefulness. As reported in Table 2-7, more than one-half of respondents scored COTR usefulness at 4 or 5 on a 5-point scale. Conversely, less than one-quarter scored COTR usefulness at 1 or 2. The details of the SBIR program are fairly complex, so a technically knowledgeable COTR has the potential to be of great use, especially to companies that are new to the program. The 2011 Survey therefore also asked respondents to share their views on the technical capacity of the COTR with regard to the SBIR program (see Table 2-8). Overall, respondents appeared satisfied; more than one-quarter indicated that their COTR was extremely knowledgeable about the SBIR program, while 3 percent indicated that the COTR was not at all knowledgeable.

TABLE 2-6 NASA SBIR: Frequency of Contact with COTRs, Reported by 2011 Survey Respondents

NOTE: N=178 Respondents.

SOURCE: 2011 Survey, Question 47.

TABLE 2-7 NASA SBIR: Usefulness of the COTR, Reported by 2011 Survey Respondents

NOTE: N=178 Respondents.

SOURCE: 2011 Survey, Question 48.

TABLE 2-8 NASA SBIR: COTR Knowledge About the SBIR Program, Reported by 2011 Survey Respondents

NOTE: N=178 Respondents.

SOURCE: 2011 Survey, Question 49.

COTRs are the project managers, but they can also provide valuable support in a number of other areas, which the 2011 Survey attempted to flesh out. COTRs sometimes also provide help in introducing awardees to technical staff at universities who could provide critical technical support. However, only about 15 percent of respondents indicated that this was the case for their project.¹⁷

COTRs are also sometimes well positioned to provide useful connections to other firms—either other SBIR awardees or other firms with complementary interests or capabilities. Just over one-quarter of respondents indicated substantial support in this area (scores of 4 or 5 on a 5-point scale) (see Table 2-9).¹⁸

COTRs may also help connect SBIR companies to specific NASA programs. Although case studies of SBIR firms suggest that COTRs can effectively provide this connection, the 2011 Survey results generally suggest that effective help is not the norm. Twenty-seven percent of respondents scored their COTR at 4 or 5 on a 5-point scale in terms of providing connections to possible markets. Over one-half scored their COTR at 1 or 2 on the same scale (see Table 2-9). It was not possible to determine what type of training COTRs receive to connect awardees to market opportunities. This may reflect the fact that, to some extent, the responsibility of making this connection is also associated with NASA’s technology infusion managers (see below for roles and activities).

In addition, the 2011 Survey asked SBIR companies about specific help received with connections to Phase III funding opportunities—Phase III emphasizing commercialization without the provision of additional SBIR funding. As reported in Table 2-10, about one-third of respondents indicated that they discussed an application in great detail with their COTR, or that their COTR provided substantial guidance during the application process, while more than 40 percent indicated that little support was provided.

__________________

TABLE 2-9 NASA SBIR: Amount of Help from COTR in COTR Help in Connecting NASA SBIR Awardees to Market Opportunities (Reported by 2011 Survey Respondents)

NOTE: N=174 Respondents.

SOURCE: 2011 Survey, Question 50.5.

TABLE 2-10 NASA SBIR: Respondent Perspective on Working with COTR on Phase III Funding (2011 Survey)

NOTE: N=177 Respondents.

SOURCE: 2011 Survey, Question 51.

The 2011 Survey also asked about the effectiveness of COTR help in acquiring Phase III funding, that is, non-SBIR funding received after the completion of SBIR Phase II. Case studies of SBIR companies suggest that COTRs have widely varied capabilities in this important area, with some focusing on the project’s scientific and technical aspects and others providing connections to the acquisition programs that will use the research results. Forty-four percent of respondents indicated that their COTR was very helpful or somewhat helpful in connecting the company to sources of Phase III funding, while 58 percent thought the COTR was not very helpful or not at all helpful (see Table 2-11).

Company Relationship with the COTR

Beyond the specific areas related to Phase III funding, the 2011 Survey also sought to determine how easy it was for SBIR companies to reach the

COTR with questions or concerns, given that COTRs usually have many other duties to manage other than the project at hand. Table 2-12 shows that more than 90 percent of survey respondents found it easy or very easy to reach their COTR. Given other duties, some COTRs may simply not have enough time to work on projects that they are supposed to be managing. However, in general, respondents did not indicate this was the case—85 percent of respondents indicated there was sufficient or more than sufficient COTR time available.¹⁹

During the case study discussions, a number of principal investigators suggested that the replacement of a COTR during the course of an award could have devastating consequences for the long-term success of the project. However, among survey respondents, only about 10 percent of COTRs were replaced during the course of Phase II awards.²⁰

TECHNOLOGY INFUSION MANAGERS

Although NASA has recently launched new initiatives that seek to connect SBIR companies with NASA opportunities, it does not provide

TABLE 2-11 NASA SBIR: Effectiveness of COTR in Connecting Awardee to Sources of Phase III Funding, as Reported by 2011 Survey Respondents

NOTE: N=151 Respondents.

SOURCE: 2011 Survey, Question 52.

TABLE 2-12 NASA SBIR: Ease with Which Respondent Could Contact COTR, as Reported by 2011 Survey Respondents

NOTE: N=177 Respondents.

SOURCE: 2011 Survey, Question 53.

__________________

dedicated agency-level support for the commercialization of SBIR technologies using third-party commercialization support companies.²¹

NASA currently relies primarily on technology infusion managers (TIMs) to make connections between SBIR companies and the acquisition programs within the Mission Directorates. TIMs are in place at all of the NASA Centers. However, NASA has not developed metrics or data about the effectiveness of TIMs. Although they possess knowledge about opportunities within their own Center, TIMs are often not well connected to other NASA Centers and do not focus on commercialization outside of NASA.

NASA has provided support to a TecFusion™ program operated by the independent Technology Commercialization Center, which seeks to link large and small businesses, although not focused exclusively on SBIR.²² The TecFusion™ program has evolved over time; it is currently on a 1-year extension of its previous contract to help NASA managers connect to companies (especially small companies) that can meet their technology needs.²³ It is a small program involving about five NASA officials.²⁴

The SBIR program office relies primarily on NASA’s Technology Transfer program to help SBIR companies find opportunities outside of NASA. However, the Technology Transfer program is not limited to SBIR. NASA did not provide data about the effectiveness of the Technology Transfer program in helping companies find commercialization opportunities either in general or specific to SBIR.

MENTORING PROGRAM

Two recent NASA initiatives have attempted to develop mentor programs for SBIR firms. One initiative arranges for large companies to mentor small companies. The other initiative pairs universities with less SBIR experience with universities with more SBIR experience.²⁵ Alabama ATT was recently assigned such a mentor. In yet another mentoring effort, NASA had encouraged tighter linkages between engineering and business students at the Georgia Institute of Technology.

All SBIR Phase II winners are automatically eligible for the Mentor Protégé Program (MPP).²⁶ (See Box 2-1.) Mentors from prime contractors receive an incentive to participate, such as a fee or a credit toward

__________________

subcontracting goals. Mentors and protégés find each other; NASA does not provide matchmaking services.²⁷ They are required to develop a formal development assistance agreement, which must commit at least 70 percent of the dollar value to technology transfer (the remaining 30 percent can be committed to business assistance). Proposed agreements must be approved at the center level and then by the Office of Small Business Programs at NASA headquarters, after which they must be formally added to contracts by the COTR. These agreements are subject to a full range of reporting requirements: semi-annual and annual reports, as well as a post hoc report. Despite the reporting requirements, the NASA SBIR office did not provide any data on take-up or outcomes for SBIR participants.

The energy and materials program at Glenn Research Center recently collaborated with a local entrepreneur support group called Launchhouse to identify NASA projects—including SBIR projects—that could be further developed in partnership with outside companies, organizations, or funders. Launchhouse worked in particular on customer validation and needs requirements, according to Matt Moran, Sector Manager, energy and materials at NASA Glenn.²⁸ Moran noted, however, that, in general, SBIR firms do not participate in the larger cooperative projects at NASA Glenn.

DATA, TRACKING, AND ANALYSIS

The NASA SBIR program exists primarily to serve NASA mission needs. However, efforts to track the extent to which these needs are, in fact, met have been limited. It is conceptually difficult to track a technology all the way from initial development to final infusion into a NASA mission or program.

__________________

Often infusion of an SBIR technology occurs many years after the award date, and the technology is a small part of a much larger system of technologies developed by different companies under different contractual arrangements. In addition, infusion often comes through a contract held by a large company in the role of NASA prime.²⁹

For the overall program, data and analysis issues can be divided into those related to data collection and those related to data utilization and analysis, as described below.

Data Collection

Outcomes Data

Until FY2012 NASA did not collect post-award outcomes data in a systematic way. NASA did not provide the committee with any data for the period preceding FY2012. More recently, NASA has begun to collect outcomes data via a new EHB module (see Chapter 3 for a more detailed description of the Electronic Handbook). The new data collection system is largely modeled on the DoD Company Commercialization Record (CCR), with some significant adjustments and improvements. The NASA system uses a more granular data structure which permits companies to enter information about multiple successes related to the same award, but according to discussions with agency staff, NASA SBIR companies are not required to certify that their records are updated before an award can be made, and, as a result, database coverage is likely not universal.³⁰

Beyond the new EHB module, NASA does not have a contract-based system for tracking Phase III awards on a systematic basis. If awards are entered into the Federal Procurement Data System (FPDS) database correctly, then it should be possible to track Phase III awards through the contracting system. However, the NASA SBIR office does not have procedures in place to ensure that contracts are entered, or that the entries are correct. Therefore, although the EHB module offers the promise of better data, it is important to ensure that sufficient priority and resources are assigned to data collection.

Process Data

In addition to tracking outcomes, tracking activities at the Field Center level would provide information central to the program. TIMs undertake a wide range of activities that vary substantially by sector, and they track these activities themselves, using a range of tools. However, there are no mechanisms for tracking the activities of TIMs more centrally.

__________________

The processes of formal NASA initiatives are also not closely tracked in some cases. For example, NASA has not been able to provide data about either the MPP take-up or outcomes.

Analytics

The limited data collection in the past has undoubtedly made it more difficult for NASA to develop a data-driven approach to program management. The data now emerging from the EHB should help to provide a basis for such an approach in the future.

Overall, NASA provided no documentation of past evaluation efforts—aside from the 2012 Economic Impact Report to suggest that the SBIR program is using process and outcomes data to drive program structure and to adjust the program based on experience. Otherwise, over the course of this study, NASA had offered to this committee little quantitative analysis of outcomes based on program data. It had provided little evidence that the program was meeting congressional goals—neither a systematic assessment of the overall take-up of SBIR technologies within NASA nor of the take-up of SBIR technologies outside of NASA via Phase III contracts.

There has been no analysis of the different approaches and resulting outcomes at the different Field Centers. These offer the potential for natural experiments as TIMs and Field Centers adopt different strategies to meet their objectives. Discussions with Field Center staff revealed that there is considerable communication among the TIMs at different Field Centers and that best practices are shared laterally among Field Centers but did not reveal similar communication between the Field Centers and the NASA SBIR office.

2012 NASA IMPLAN Assessment

In 2012 NASA hired a consultant to provide an economic impact assessment of the NASA SBIR program, based on an input-output framework and multipliers, and implemented using the IMPLAN input-output methodology³¹ and software. A variety of agencies have used IMPLAN to assess the expected effects of a program change on measures such as employment, income, output, and taxes.

Although this mode of analysis can generate some interesting results, it is not an effective approach to assessing the extent to which the NASA SBIR program meets congressional objectives, because none of those objectives is measured using input-output analysis. Congressional objectives do not include jobs, economic growth, or return on investment. Furthermore, the assessment was not designed to provide formative guidance on process improvement—another useful application of program evaluation. However, the results have

__________________

been used for program accountability and to meet Administrative and Congressional requirements that federal agencies evaluate their programs.

NASA FIELD CENTERS AND SBIR

Each of the NASA Field Centers has an SBIR program that to some degree reflects its unique capabilities and history. To provide some insight into the differences among the Field Centers, this section addresses the activities of TIMs at Langley Research Center, Johnson Space Center, and the Jet Propulsion Laboratory.

Langley Research Center (LaRC)

LaRC was founded in 1917 as the nation’s civil aeronautics laboratory, with competencies in aeronautic research, testing, and mechanical fabrication, and atmospheric and space science. Its SBIR/STTR annual portfolio averages 350 active Phase I, Phase II and Phase II enhancement projects.

LaRC SBIR/STTR infusion is driven by early identification of Mission Directorate customer needs, expressed in various NASA program and technology guidance—plus SBIR topics/subtopics—and by the need to connect these needs effectively with LaRC Engineering Directorate staff.

LaRC has developed new information technology (IT) infrastructure on which to build systems that can link Mission Directorate clients, LaRC engineering staff, and SBIR/STTR companies. By adapting a Salesforce™ “Sales Process Map” to Microsoft Project software, the LaRC TIM developed an IT backbone and database. The TIM then enriched the database with NASA customer information from the National Research Council’s³²Visions and Voyages for Planetary Science in the Decade 2013–2022 report,³³ NASA’s Technology Area Roadmaps, and the Aeronautics Research Plan. This enabled the TIM to map the LaRC SBIR program design to NASA Strategic Goals, LaRC’s overall vision, and Strategic Focus Areas. Finally, the TIM devised specific SBIR infusion goals and objectives, which are aligned with Mission Directorate technology priorities.

The TIM markets SBIR technologies and companies to potential NASA customers through periodic web-based “Innovation Updates” on SBIR projects. The TIM also provides NASA customers with more detailed information about the benefits of Phase III SBIR contracting.

The TIM works to attract key Engineering Directorate staff to serve as COTRs. In particular, the TIM has enhanced COTR participation in Phase I and

__________________

II project close-out discussions, as well as Phase II project midterm reviews, which are led by topic/subtopic sponsors from the Mission Directorates. Creating a solid personal and professional link between Mission Directorate and engineering staff is a core characteristic of LaRC infusion strategy.

Outside NASA, an external partnership management plan supports industry/university/laboratory partnerships with SBIR projects. The plan includes outreach via social and conventional media, and participation in regional small business promotional events.

Now that the IT tools are in place, the TIM has implemented an annual process improvement review to determine progress toward these goals and has made some program course corrections as a result. Key metrics cover Phase I and II proposal submissions, Phase II enhancement activity, and Phase III contracts; however, these data were not provided to the committee.

Johnson Space Center (JSC)

JSC opened in 1961 as NASA’s primary center for design, development, and testing of spacecraft and associated systems for human flight. Unique among the Field Centers, JSC is owned and operated by the nonprofit Manned Space Flight Education Foundation, with a high-priority NASA public education mission that extends to its SBIR program. Historically, JSC has been at the heart of NASA outreach, which has also affected SBIR activities, as indicated below.

A range of pathways are employed to connect Mission Directorates, engineering staff, and SBIR companies. For example, TIMs participate in Mission Directorate activities at the Field Center and in Mission Support Division activities for the Mission Directorates. Participation in these activities generates detailed information about customer needs and demands, which the TIM can then pass on to SBIR companies.

Externally, the SBIR program actively piggybacks on Field Center regional outreach. The TIM participates in local/regional “speed-dating” events and small business technology markets with industry partners. In addition, while JSC has reached out to scientists and engineers at regional Historically Black Colleges and Universities (HBCUs) and Minority Serving Institutions (MSIs) by providing onsite technical briefings about NASA programs and Mission Directorate roadmaps, the TIM has provided professors and graduate students with information about SBIR as a pathway into NASA. It is notable that the TIM uses Phase I applications from HBCUs/MSIs as a metric for success.

The TIM also collaborates with other agencies’ SBIR programs, such as the Missile Defense Agency and the Defense Advanced Research Projects Agency (DARPA), to expand customer opportunities, leverage SBIR funding, and provide technical assistance for SBIR awardees. Collaboration with foreign space programs, including Japan Aerospace Exploration Agency (JAXA), is designed to expand customer opportunities for SBIR projects.

Jet Propulsion Laboratory (JPL)

JPL is managed for NASA by the California Institute of Technology (Caltech) and is the leading U.S. center for robotic exploration of the solar system, focused on spacecraft payloads (not on rockets themselves).

JPL is also NASA’s only Federally Funded Research and Development Center (FFRDC); unlike NASA’s other nine field centers, which are staffed by government civil servants, JPL staff—including SBIR project technical monitors—consists almost entirely of Caltech researchers and engineers.

The JPL SBIR infusion strategy differentiates between near-term practices and longer-term customer (NASA program) engagement:

• Near-term infusions are purely technical and may be linked to Mission Design Reviews. They are focused on SBIR technology at Technology Readiness Level (TRL) 6 or higher.
• Longer-term infusions have less direct impact on mission programs. According to JPL staff, impacts include contribution to technical state-of-the-art or application to a priority technical trade space.

The TIM participates in bi-weekly meetings of the JPL Technology Working Group (TWG), a high-level strategic planning body at the Division/Directorate level. TWG’s work aims to ensure that the SBIR program is aligned with JPL mission priorities, and it also allows senior staff input into and influence over topic/subtopic development, while strengthening the visibility of SBIR projects for senior Caltech scientists and engineers.

The TIM leverages the skills and connections of veteran JPL engineers by recruiting them as technical monitors (TMs):

• SBIR is positioned as an incentive to JPL engineers—it provides access to leading-edge research and development in exchange for limited hours of TM service.
• TMs are encouraged to communicate NASA mission requirements and the mission culture of specific programs to SBIR awardees.

JPL staff were reluctant to discuss further details of program management or of individual SBIR projects.

INTELLECTUAL PROPERTY AND DATA RIGHTS

The SBIR program is clear that the intellectual property (IP) rights developed under an SBIR award remain the property of the company.³⁴ Although the government—as with all federal contracts—retains “march in”

__________________

rights whereby it can use the technology for its own purposes, the stated requirements of the program insist that companies retain all IP rights for a period of 4 years after the end of the Phase II award. The SBA website on SBIR data rights states: “SBIR/STTR Data are protected from disclosure by the participating agencies for a period of not less than 4 years from delivery of the last deliverable under the Phase I, II, or III award. The protection period is extended with each subsequent related award in order to avoid harmful disclosure of SBIR/STTR data related to on-going federally funded SBIR/STTR efforts.”³⁵

The SBA Policy Directive defines IP very broadly, explicitly not limiting this definition to patents only: “The separate and distinct types of intangible property that are referred to collectively as ‘intellectual property,’ including but not limited to: (1) patents; (2) trademarks; (3) copyrights; (4) trade secrets; (5) SBIR technical data (as defined in this section); (6) ideas; (7) designs; (8) know-how; (9) business; (10) technical and research methods; (11) other types of intangible business assets; and (12) all types of intangible assets either proposed or generated by an SBC as a result of its participation in the SBIR Program.”³⁶

NASA’s approach to SBIR contracts rests, however, more on statutory regulations related to contracts, and in particular on provisions drawn from the Bayh-Dole act that focus on patents and patenting. NASA contracts require that all inventions be disclosed within 2 months, preferably via the agency’s New Technology Reporting website.³⁷ According to NASA, registration provides the company with a free worldwide license to the technology. Failure to register is a violation of FAR 52.227-11 and can result in loss of IP protections.

This tension between SBIR regulations and federal contracting could lead to circumstances in which companies decline to seek patent protection and NASA subsequently steps in to patent an invention, potentially over-riding the data protection provisions of the SBIR legislation. However, companies contacted for case studies indicate that this scenario is, in practice, quite unlikely. Although there is a tension between the provisions, in practice this is unlikely to cause damage to an SBIR company.

__________________