National Academies Press: OpenBook

SBIR at the Department of Defense (2014)

Chapter: 6 Program Management

« Previous: 5 Insights from Case Studies and Extended Survey Responses
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

6

Program Management

As with any major government program, the relative success of different Small Business Innovation Research (SBIR) programs at different components and Services depends to a considerable degree on program implementation and program management. In this chapter, the committee undertakes a detailed review of how various services and components have sought to improve the SBIR program’s efficiency in recent years. In particular, this chapter explores issues related to the selection of topics, which guide the technical direction of the program; source selection, and which determines which applicants receive funding; and a range of other SBIR process issues, including an extended discussion of Technical Points of Contact (TPOCs), the liaisons between components and companies. The committee also reviews efforts to encourage the participation of women and minorities within the DoD SBIR program. . Data for this chapter is drawn primarily from the National Research Council (NRC) 2011 survey, agency interviews, agency documents, and case studies along with workshops. These sources are discussed further in Appendix A.

TOPIC SELECTION

As DoD has become more focused on improving outcomes from SBIR programs, concerted efforts have been made to ensure that agency needs are clearly defined, that topics match those needs, and that components utilize the results of research conducted with SBIR funding.

In this section, the committee focuses on the process used at Navy, and in particular the Naval Sea Systems Command (NAVSEA).1 Navy has made

__________________

1For an earlier review, see National Research Council, “Review of the Naval Sea Systems Command (NAVSEA) Draft Memorandum, “NAVSEA’s 21st Century Engagement, Education, and Technology Initiative,” Washington, DC: The National Academies Press, 2008. This report notes that “NAVSEA recognizes the critical challenge of maintaining an adequate number of skilled

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

available documents describing this process in some detail, but this presentation is not a claim that NAVSEA processes are best practice.2

Establishing Agency Needs

The first step in the development of topics at Navy is to define technical needs. Navy Program Executive Offices (PEOs) and Headquarters (HQ) directorates are invited to prepare R&D needs statements. These must address clearly identified technological gaps in critical Navy Research and Development (R&D) or acquisition programs, as well as other Navy objectives. Each submission must meet a number of technical requirements and is presented in the form of a quad chart that includes objectives, needs, references, and appointed contact.

All of the submitted needs are collated by the Navy SBIR program office and are then submitted for review and eventual approval to the Navy SBIR Board of Directors, a committee of flag-level representatives from each PEO and system command within NAVSEA.

The Board of Directors meets at the beginning of each SBIR topic cycle year. It reviews and then selects the needs against which topics can be developed. Only needs approved by the board can be used to justify the topic. Use of a Board of Directors structure also encourages the senior staff to buy into the SBIR program, because it gives them a clear role in directing the program toward their own technical needs and interests.

Topic Development

Once the Navy’s technical needs have been established for the upcoming topic development cycle, topic authors from the PEOs and HQ directorates can prepare topics to meet those needs. The broad process of topic development is captured in Figure 6-1.

Topics are developed by technical staff at PEOs and HQ directorates and are then submitted for initial review by the NAVSEA SBIR program manager. After possible revision and eventual approval, topics are then reviewed and approved by the Board of Directors before final submission to the Navy SBIR program office for solicitation approval. They are then forwarded to the Office of Small Business Programs (OSB) for review. In some cases, DoD review rejects proposed topics—most recently, according to Navy staff, because

__________________

Science, Technology, Engineering and Mathematics (STEM) professionals. NAVSEA correctly reasons that this challenge has implications for its own workforce development, and its commitment to address this challenge as it relates to its own mission is commendable.”

2This section is drawn from NAVSEA, “NAVSEA SBIR Needs and Topic Validation Process,” April 2012, and from interviews with Dean Putnam, NAVSEA SBIR Program Manager; Dr. Regan Campbell, PEO SUBS Director for Undersea Warfare; Glen Sturtevant, PEO Ships SBIR Director; and Douglas Marker, PEO IWS SBIR Director, June 25, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

images

FIGURE 6-1 SBIR topic selection: Current practices at Navy (May 2012).
NOTE: See Appendix D for all DoD acronyms.
SOURCE: Navy SBIR Office.

the technologies they call for are not sufficiently innovative,3 although Bonnie Heet, SOCOM SBIR Program Manager, noted that these rejections tend to result more from topic duplication.4

Aside from this fairly elaborate procedure, NAVSEA uses a well-documented set of criteria to assess and validate topics (see Box 6-1). In addition to setting out detailed criteria, NAVSEA works to enhance the quality of topics by training topic authors. All authors are required to go through topic training provided by the NAVSEA SBIR program office. NAVSEA also provides a “prescription” for writing a quality topic (see Box 6-2). Meeting these requirements helps to ensure topic quality, according to Dean Putnam, NAVSEA SBIR Program Manager.5

Although each of the components and Services has its own process for writing topics, and these are likely to differ significantly from the NAVSEA process described above, the general point is that the era in which topic writing was left largely to individual authors, and in which there was almost no linkage between acquisition offices and topics, has largely ended. At the Special Operations Command (SOCOM), for example, topics must be accompanied by a transition plan. In interviews with Air Force (AF) staff, it became clear that

__________________

3See NAVSEA interviews, note 1.

4Interview, Bonnie Heet, SOCOM SBIR Program Manager, June 28, 2013.

5Interview, Dean Putnam, NAVSEA SBIR Program Manager, June 25, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 6-1

NAVSEA Topic Selection Criteria

  1. All required fields of the topic template must be completed.
  2. Topics must clearly articulate the Board of Directors-approved Navy need and its applicability to the requested technology.
  3. Topics must clearly identify and discuss the needed innovation and R&D. Topics must discuss currently available technology, both Government and commercial, and its applicability to the needs identified in the topics.
  4. Topics must be technically clearly written so that they are understandable to small business personnel.
  5. Topics must be congruent with SBIR funding levels for Phases I and II.
  6. Target transition programs must be identified (R&D or Acquisition).
  7. Topics must not require secure access for Phase I.
  8. Topics must not contain classified information.
  9. Grammar, spelling, and clarity are important and, if not addressed, may disqualify a topic. Acronyms must be spelled out and Government jargon must be avoided.
  10. Topics must not duplicate one another.
  11. Topic references must be publicly accessible and available; and two to four must be provided.
  12. Topics must identify high-level technical requirements.

____________________________

SOURCE: NAVSEA: NAVSEA SBIR Needs and Topic Validation Process, April 2012.

connections with acquisition offices are a high priority: According to AF staff, DoD is now targeting PEO sponsorship for more than 50 percent of topics.6

Topic Specialization

One possible area for further innovation in program design may be specialization. For example, SOCOM (which serves Special Forces) tends to focus on needs that are unique to its user base—very lightweight, highly durable, rugged technologies that can be delivered quickly.

In this case, most topics are focused on standalone technologies that are not part of larger weapons systems and hence can be ported to use by other DoD

__________________

6Interview with Dr. Leslie Perkins, David Sikora, and Richard Flake, AF SBIR Program, June 28, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 6-2

NAVSEA Prescription for Quality Topics

  • Describe the Navy problem to be solved and cite the Navy need identified by the Board of Directors and its relevance to the topic;
  • Define the current state-of-the-art of technology available today, both commercial and Government, and its applicability to the need described in the topic—the current state;
  • Describe where the Navy needs to be to solve the problem—the future state;
  • Identify and describe the technological challenges encountered in bridging the gap between the current and future states;
  • Describe the innovation and/or R&D needed to address the technological gaps; and
  • Provide sufficient information to guide small businesses in developing high-quality proposals.

SOURCE: NAVSEA: NAVSEA SBIR Needs and Topic Validation Process, April 2012.

components relatively easily. SOCOM targets a Technical Readiness Level (TRL)7 of 6 or 7 at the end of Phase II—somewhat closer to combat readiness than SBIR awards at other components, which often need to be integrated by developers of larger systems. And as a result, SOCOM relies less on prime contractors to deliver weapons systems, offering more opportunities for SBIR winners to move on directly to contracts with SOCOM.

Because SOCOM has a relatively small SBIR budget (about $10 million) it is highly motivated to find synergies with bigger programs elsewhere. Therefore, it has also developed systems for disseminating white papers received from potential applicants as a means of gauging interest from other potential funders.

The smaller size and focused target audience also makes it easier for SOCOM SBIR to connect to the wider SOCOM supply chain: a single national conference (SOFIC) is held in conjunction with the National Defense Industry Association (NDIA) every year, attracting 7,000-8,000 attendees. Thirty-four SOCOM SBIR contractors attended in 2012, and 14 had booths.8

__________________

7TRL: Technology Readiness Level. This is a widely used metric within DoD to describe the state of readiness of a particular technology. Technologies ready for use by warfighters are at TRL 8-9. Basic research is TRL 1-2.

8Bonnie Heet interview, note 4.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

SOURCE SELECTION

The selection of specific proposals for awards is known as “source selection” within DoD. It is tightly governed by detailed federal contracting rules. These define, for example, the release of information in applications, who is permitted to view application details, and nondisclosure and conflict-of-interest requirements among evaluators and selection officers.9 Information flow even after an award is made is also tightly controlled.10

Applications details are specific to each component. At Navy, even Phase I proposals must include a commercialization plan. In all cases, they must address the topic, provide evidence that the firm is capable of completing the research successfully, offer at least a brief description for the eventual take-up of the technology, include a detailed budget, and, if possible, indicate interest from PEOs or other potential downstream sponsors.

Currently, Army and Navy are working to focus on the most promising technologies as quickly as possible. Therefore, all their applicants must include an “option” plan for bridging the gap between the end of Phase I and the possible start of a Phase II. AF does not call for an option plan. As a result, Phase I funding is limited to $80,000 at Army and Navy and $150,000 at the Air Force.11 Once received, the application is subject to a fairly elaborate selection process.

Phase II selection is similar, but with increased emphasis on commercialization. For example, at Navy “Phase II SBIR Proposal Invitation forms must identify the Navy acquisition or R&D program that will potentially transition the contractors’ SBIR technology, product or service to Phase III and into the acquisition process. In addition, the NAVSEA SBIR PO requires from the acquisition or R&D program a Transition Memo indicating the desire to proceed with the contractor into Phase II, the potential for Phase III funding, and the requirements to be met by the contractor during SBIR Phase II development.”12 At AF, recent changes in the program—notably the introduction of a shared-cost Phase II.5 and the introduction of new liaison positons between companies and acquisitions - have strengthened the link between Phase II funding and acquisitions. 13 (Note however that under the 2011 reauthorization legislation, DoD components are no longer permitted to

__________________

9Federal Acquisition Regulations (FAR) can be found online at the FAR home page, <http://www.acquisition.gov/far/>, accessed July 11, 2014.

10NAVSEA policy notes that “Post source selection discussions may be prohibited by 15 USC §638, the Privacy Act (5 USC §552a), the Trade Secrets Act (18 USC §1905) and other laws.” NAVSEA, “Source selection process small business innovation research (SBIR) program Phase I and Phase II awards,” February 2012, p. 4.

11DoD SBIR Solicitation FY 13.2.

12NAVSEA, February 2012, note 10, p. 18.

13Air Force, NAS Briefing on SBIR/STTR Program, June 28, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

selectively invite firms to apply for Phase II—all Phase I winners are permitted to do so).14

Composition of Evaluation Panels

Source-selection decision makers are a pivotal point in the selection process, so the composition of selection panels is an important question. At DoD, it appears that all components use only DoD personnel. All components appear to employ a selection panel of two or three evaluators. Some components use a lead evaluator, who is charged with both managing the application through the process and resolving conflicting views among the evaluators.

At NAVSEA, a panel is established for each topic, with three evaluators and a chairman, selected by the technology manager for the topic. Each panel must include a topic expert, a Research and Systems Engineering (R&SE) expert, and a mission area expert (the latter two must be from outside the sponsoring program office, and the third must be a government employee).15

Selection Criteria

All components must address the selection criteria defined by DoD, which are published in the solicitation itself (see Box 6-4). Different components may score proposals differently. At Navy, scoring is as follows:16

(a) Technical Merit—40 points

(b) Qualifications of Key Personnel—30 points

(c) Potential for Commercialization—30 points

Even though applications may require a commercialization plan—and even though commercialization is directly part of the rubric for scoring applications—selection panels are not required to include a member with commercial business expertise.

Selection: Conclusions

The difficulty of selecting from the numerous high-quality proposals also makes it difficult to ensure that all Congressional objectives are met in the course of the selection process.

1)   Commercialization. The recent attention paid to this issue—illustrated in this section and others—means that selection is now heavily focused

__________________

14See Chapter 1 “Introduction,” which contains a summary of the reauthorization legislation.

15NAVSEA, February 2012, (above, p. 9).

16NAVSEA, February 2012 (above, note 9), p. 12.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 6-3

DoD SBIR Phase I Selection Criteriaa

“The DoD Components plan to select for award those proposals offering the best value to the Government considering the following factors which are listed in descending order of importance, unless otherwise stated in the Component’s instructions in Section 8.0 of this solicitation.

(a)   The soundness, technical merit, and innovation of the proposed approach and its incremental progress toward topic or subtopic solution.

(b)   The qualifications of the proposed principal/key investigators, supporting staff, and consultants. Qualifications include not only the ability to perform the research and development but also the ability to commercialize the results.

(c)   The potential for commercial (Government or private sector) application and the benefits expected to accrue from this commercialization.

Firms with a Commercialization Index Achievement (CAI)b score at the 20th percentile or below may receive no more than half of the evaluation points available for commercial potential criteria (see Section 3.5.d).c Where technical evaluations are essentially equal in merit, cost to the Government will be considered in determining the successful offeror.”

a Formal selection criteria for Phase II are usually identical to those for Phase I, although more attention is paid to commercialization plans.

b The CAI attempts to measure the extent to which a company’s SBIR and STTR awards have resulted in commercial activity. See DoD SBIR Desk Reference, Section II—Evaluation and Selection. <http://www.acq.osd.mil/osbp/sbir/sb/resources/deskreference/02_eval.shtml>. Accessed August 15, 2013.

c Thus, at NAVSEA, for example, firms with a CAI falling into the bottom quintile can score a maximum of 15 out of 30 for commercialization potential.

 

SOURCE: DoD SBIR Program Solicitation FY 13.2 Section 6.0.

on transition: ensuring that the technologies developed through the SBIR program find their way into use in DoD. This emphasis is demonstrated in the outcomes described in other parts of this report.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      Use of the CAI to penalize the lowest-performing score likely enhances commercialization impacts.17

2)   Agency mission. DoD components have made extensive efforts in recent years to align SBIR topics, awards, and hence outcomes with clearly defined agency needs—as demonstrated by the detailed process used by NAVSEA and described above.

3)   Knowledge effects. DoD identifies the “innovation” of a proposal as part of the most important criterion. In addition, there is evidence that the DoD-level review of topics increasingly focuses attention on the need for innovation.

4)   Women and minorities. There is no evidence that this Congressional objective plays a role in the selection processes at any DoD component.

It should be understood that in the context of DoD, “commercialization” takes on a special meaning. In general usage, technology commercialization reflects efforts to reach the market and is usually measured by the extent to which such efforts are successful.18 Commercialization is also a process normally viewed at least in part from the perspective of the company: private firms that do not “commercialize” by generating revenues from the sale of products or processes eventually vanish.

“Commercialization” means something quite different at DoD, where it is widely taken to mean the acquisition of SBIR-funded technologies by end users within DoD—especially by programs of record or other formal acquisition programs within DoD. This issue is discussed in Chapter 4 and in Appendix A.

OTHER SBIR PROCESS ISSUES

Following completion of the first round of NRC analysis and publication of the related reports, the current assessment sought to identify additional information about the process of implementing SBIR awards, with a view to providing management with more detailed information about program operations. This section considers several operational aspects of the program.

Matching Funds

In general, DoD SBIR awards do not require matching funds for Phase I or Phase II. However, it is possible that some components favor proposals that include matching funds at this stage. The recent NRC survey addressed this question: 138 DoD Phase II respondents (18 percent) indicated that they had

__________________

17Although it should also be noted that, although the CAI score is used to identify and down-score the bottom quintile of firms with at least four previous Phase II awards, it is not apparently used in a systematic way to help differentiate among the 80 percent of firms who score above the 20th percentile.

18See also the discussion of commercialization in Chapter 4.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

matching funds.19 These matching funds came from a variety of sources. Table 6-1 shows the frequency with which each source was reported by these Phase II respondents.

The source mentioned most often by respondents was non-SBIR federal funding, followed by company funds. Venture Capital (VC) and angel funding together accounted for about 10 percent of responses. Non-SBIR federal funding at DoD is closely aligned with investments from programs of record and other acquisition sources. They can also be aligned with funding from DDR&E (Director of Defense Research and Engineering).

Although matching funds are a very positive signal of long-term interest in a project, adding requirements in this area could constitute a significant burden for small firms. Almost 40 percent of Phase II respondents indicated that it took no more than 2 weeks of full-time effort to find the funds, but 25 percent indicated that it took at least 2 months of equivalent effort (see Table 6-2).

Evidence from case studies adds context to these data. Interviewees suggested that additional funds could be acquired relatively easily by firms that already had funding agreements in place (e.g., partnerships with larger companies), but matching funds requirements could in other cases present a significant hurdle or at least delay.

Uses of matching funds beyond the initial Phase II award (e.g., Navy’s Phase 2.5 program) are discussed in Chapter 4.

TABLE 6-1 Sources of Matching Funds for Phase II Awards for Those Reporting Some Matching Funds

Phase II Respondents (Percent)
Federal non-[SBIR/STTR] funding 40.4
Our own company (includes borrowed funds) 33.8
Another company 27.9
An angel or other private investment source 7.4
Venture capital 2.2
Other (please specify) 16.2
N = 136

NOTE: Because respondents could choose more than one category, responses do not sum to 100 percent.
SOURCE: 2011 NRC Survey, Question 28.

__________________

19Matching funds were sought by components only for Phase II awards, so only Phase II respondents were asked this question.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-2 Time Needed to Acquire Matching Funds

Phase II Respondents (Percent)
No additional effort needed except paperwork 15.3
Less than 2 weeks Full-Time Equivalent (FTE) for senior company staff 23.7
2-8 weeks of effort FTE for senior company staff 35.9
2-6 months of effort FTE for senior company staff 19.1
More than 6 months of effort FTE for senior company staff 6.1
Total 100.0
N = 131

SOURCE: 2011 NRC Survey, Question 29.

Funding Gaps

Agencies have become much more attuned to problems caused by funding gaps, which can be especially challenging for small firms because they are less likely to have access to other funding sources to keep projects on life support until Phase II funding arrives.

Both Army and Navy include an option to cover the Phase I-Phase II gap in their standard SBIR application process. Companies are required to describe how they would utilize a $70,000 award to cover the gap (which can be taken up by the agency when it decides that Phase II funding will be awarded).20

Despite these agency initiatives, 71 percent of Phase II respondents indicated that they experienced a gap between the end of Phase I and the start of Phase II for the surveyed award (see Table 6-3). The data in this table are essentially identical to those for the NRC’s 2007 survey (69 percent), which suggests that these initiatives have not yet had the desired impact.21 This gap had a range of consequences for the company. Table 6-4 indicates the types of impact on respondents who experienced a funding gap.

TABLE 6-3 Funding Gap Between Phase I and Phase II

Phase II Respondents (Percent)
Yes 70.9
No 29.1
100.0
N= 763

SOURCE: 2011 NRC Survey, Question 22.

__________________

20See DoD SBIR Program Solicitation FY 13.2.

21National Research Council, An Assessment of the SBIR Program at the Department of Defense, C. W. Wessner, ed., Washington, DC: The National Academies Press, 2009, p. 264.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-4 Effects of Funding Gaps between Phase I and Phase II

Phase II Respondents (Percent)
Stopped work on this project during funding gap 66.8
Continued work at reduced pace during funding gap 23.2
Continued work at pace equal to or greater than Phase I pace during funding gap 2.4
Received bridge funding between Phase I and II 4.6
Ceased all operations during funding gap 0.9
Other (please specify) 2.0
Total 100.0
N= 539

SOURCE: 2011 NRC Survey, Question 23.

Two-thirds of respondents reported that they stopped work during this period, while a large majority of the remainder worked at a reduced level of effort. About 1 percent ceased operations. These responses indicate a slight worsening of effects compared to the 2005 NRC Survey, which reported that 58 percent stopped work on the project.22

Aside from the direct impact of delayed projects, funding gaps can have long-term consequences, especially for smaller companies, for which there could be insufficient work to retain key project staff during the gap period. Several companies interviewed for case studies noted these dangers.

Ease of Application

The NRC survey also sought to probe more deeply into the process of SBIR application and award management. One question concerned the degree of difficulty involved in applying for a Phase II award compared with applying to other federal programs.

About 40 percent of respondents reported that the SBIR Phase II application process was easier than the process for other federal funding, while about 9 percent of respondents indicated that it was harder (see Table 6-5). These results suggest that the process does not impose undue burdens on applicants.

Funding

Although there are obvious limitations to the utility of asking recipients whether the amount of money provided was sufficient for the project at hand, there is at least some value in determining the extent of positive responses.

__________________

22Ibid., p. 264.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-5 Ease of Application for SBIR Phase II Award at DoD

Phase II Respondents (Percent)
Much easier than applying for other federal awards 11.3
Easier 30.4
About the same 40.7
More difficult 6.9
Much more difficult 2.1
Not sure, not applicable, or not familiar with other federal awards or funding 8.6
Total 100.0
N= 759

SOURCE: 2011 NRC Survey, Question 41.

In this case, about 55 percent of Phase II recipients indicated that they received sufficient funding; about 45 percent indicated that more funding was required. Less than 1 percent reported that they received more funding than necessary (see Table 6-6).

Although awardees often suggest in other contexts (e.g., case study interviews) that the size of awards should be increased (a view especially prevalent before the increases made in recent reauthorization), the survey asked about the possible trade-off between the size of awards and the number of awards. Unless agency funding for SBIR programs increases overall, larger awards inevitably imply fewer awards. In the context of that trade-off, there was no clear majority for (or against) an increase in the size of individual awards (see Table 6-7).

The survey also asked about the possible expansion of the SBIR program itself. Perhaps not surprisingly, about 70 percent of respondents indicated that they would support an increase in the size of the program even if funding were taken from other federal programs that they value (see Table 6-8).

WORKING WITH PROJECT MANAGERS

Interviews with awardees and even agency staff reveal that one of the critical factors affecting the success of individual SBIR projects is the relationship between the awardee and the agency’s project manager (at DoD, the latter is usually called the technical point of contact, or TPOC).

The survey asked a series of questions aimed at identifying ways in which this relationship might be improved. The committee hypothesized that there might be a wide variation in the degree to which TPOCs actually engage with their awardee projects, because there appears to be no DoD-wide standard for this.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-6 Adequacy of Phase II Funding

Phase II Respondents (Percent)
More than enough 0.4
About the right amount 55.4
Not enough 44.2
Total 100.0
N= 758

SOURCE: 2011 NRC Survey Question 42.

TABLE 6-7 Views on Trade-off of Larger Awards for Fewer Awards

Should the size of Phase II awards be increased even if a proportionately lower number of Phase II awards are made? Phase II Respondents (Percent)

Yes

36.1

No

38.8

Not sure

25.1

Total

100.0
N= 761

SOURCE: 2011 NRC Survey, Question 43.

TABLE 6-8 Increasing the Size of the SBIR Program

Recommendations that the size of the SBIR program be… Phase II Respondents (Percent)
Expanded (with equivalent funding taken from other federal research programs that you benefit from and value) 72.0
Kept at about the current level 26.1
Reduced (with equivalent funding applied to other federal research programs you benefit from and value) 0.8
Eliminated (with equivalent funding applied to other federal research programs you benefit from and value) 1.2
100.0
N= 763

SOURCE: 2011 NRC Survey, Question 44.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

Respondents were asked about the frequency of their engagement with their TPOC. A majority reported monthly contact, while 30 percent reported quarterly contact (see Table 6-9).

Interviews indicated that some TPOCs had very positive effects on their awardee companies, while others were of limited help. The survey attempted to gauge impacts by asking respondents how helpful the TPOC had been to their project (see Table 6-10).

Overall, more than one-half of Phase II respondents scored TPOC usefulness at 4 or 5 on a 5-point scale. Conversely, less than one-quarter scored usefulness at 1 or 2 on the scale. Phase II respondents were more likely to score usefulness at 4 or 5, while Phase I respondents were more likely to score it at 1.

Given the significant and ongoing turnover in TPOCs, one important question is the extent to which TPOCs are able to provide technical advice to the awardee about the operations of the SBIR program. In detail, the program is fairly complex, so a technically knowledgeable TPOC can be of great use, especially to companies that are new to the program.

The survey therefore asked respondents to indicate their views on the technical capacity of the TPOC with regard to the SBIR programs. Overall, more than 65 percent reported that their TPOC was extremely knowledgeable or quite knowledgeable about the SBIR program. Only about 4 percent reported that their TPOC was not at all knowledgeable (see Table 6-11).

Because TPOCs are the technical point of contact at the agency, they should be technically knowledgeable about the science and engineering involved in the award and, therefore, should be expected to provide valuable direct insights in some cases. About one-third of respondents reported receiving a substantial amount of technical help from the TPOC (scores of 4 or 5 on 5-point scale). However, about 40 percent received low or very low levels of help, which suggests that components should reconsider their expectations for TPOC performance in this area (see Table 6-12).

TPOCs are also sometimes well positioned to provide useful connections to other firms—either other SBIR awardees or other firms with complementary interests or capabilities. These connections may be especially important at DoD, where so much of the acquisitions process runs through the prime contractors. Slightly less than 30 percent indicated substantial support in this area (scores of 4 or 5 on a 5-point scale) (see Table 6-13). Therefore, it does not appear overall that TPOCs prioritize networking on behalf of SBIR awardees as part of their role.

TPOCs may serve as a critical liaison to programs of record and other possible markets for SBIR-funded products and services. Reports from interviewees were mixed: some TPOCs went to great lengths to provide this connection, while others were of little help, often because the original TPOC moved to a new job (see further discussion below).

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-9 Frequency of Contact with TPOCs

Phase II Respondents (Percent)
Weekly 13.2
Monthly 52.3
Quarterly 29.7
Annually 4.8
Total 100.0
N = 757

SOURCE: 2011 NRC Survey, Question 47.

TABLE 6-10 Usefulness of the TPOC

Phase II Respondents (Percent)
Invaluable (5) 23.9
4 34.1
3 23.5
2 13.7
No help (1) 4.8
Total 100.0
N= 757

SOURCE: 2011 NRC Survey Question 48.

TABLE 6-11 TPOC Knowledge about the SBIR Program

Phase II Respondents (Percent)
Extremely knowledgeable 23.3
Quite knowledgeable 43.5
Somewhat knowledgeable 29.0
Not at all knowledgeable 4.2
100.0
N= 756

SOURCE: 2011 NRC Survey, Question 49.

TABLE 6-12 TPOC Technical Support for Project During Phase II

Phase II Respondents (Percent)
Most helpful (5) 12.9
4 21.6
3 24.1
2 20.6
Least helpful (1) 20.8
Total 100.0
N= 744

SOURCE: 2011 NRC Survey, Question 50.2.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-13 TPOC Connections to Other Private Firms

Phase II Respondents (Percent)
Most helpful (5) 11.0
4 18.7
3 18.4
2 21.4
Least helpful (1) 30.5
Total 100.0
N= 738

SOURCE: 2011 NRC Survey, Question 50.4.

Overall, about one-quarter of respondents scored their TPOC at 4 or 5 on a 5-point scale in terms of providing connections to possible markets. Conversely, more than one-half scored their TPOC at 1 or 2 on the same scale (see Table 6-14).

In addition, the survey asked about specific help with connecting to Phase III funding opportunities. About 40 percent of respondents discussed this connection with their TPOC or the TPOC provided a lot of guidance during the process, while slightly less than 40 percent received no help (see Table 6-15).

These findings suggest that some TPOCs are well connected to acquisitions, but many others are not. Given the importance of Phase III for agency objectives, this may be an area for future DoD review.

The survey also asked about the effectiveness of TPOC guidance in acquiring Phase III funding. Interviewees highlighted this as an important issue, because TPOCs displayed widely varied capabilities, with some being better with the scientific and technical aspects of the project and others with connections to the acquisition programs. About 45 percent of respondents thought their TPOC was very helpful or somewhat helpful in connecting the company to Phase III funding sources, while about 55 percent thought the TPOC was not very helpful or not at all helpful (see Table 6-16).

TABLE 6-14 TPOC Help in Connecting SBIR Awardees to Market Opportunities

Phase II Respondents (Percent)
Most helpful (5) 10.1
4 16.3
3 21.9
2 20.3
Least helpful (1) 31.4
Total 100.0
N= 743

SOURCE: 2011 NRC Survey Question 50.5

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-15 Working Closely with TPOC on Phase III Funding

Phase II Respondents (Percent)
The officer provided a lot of guidance during the application process 14.2
We discussed the application in detail 26.3
Not much 20.8
Not at all 17.6
We did not apply for Phase III funding 21.1
100.0
N= 754

SOURCE: 2011 NRC Survey, Question 55.

Contacting the TPOC

Beyond the concerns specific to Phase III funding, the survey also sought to determine the ease with which companies could reach the TPOC with questions or concerns, given that TPOCs have many other priorities to manage. Table 6-17 shows that about 90 percent of respondents found it very easy or easy to reach their TPOC.

Several case study interviewees explained that the replacement of a TPOC during the award period can have devastating consequences for the long-term success of the project. TPOCs often serve as the project’s champion within the funding agency and as the primary link or liaison to acquisition programs and other sources of further funding. Approximately one-third of Phase II respondents reported that their TPOC for the surveyed project was replaced during the award period (Table 6-18). Although not surprising, this is an important finding that should be the subject of further review by DoD.

TABLE 6-16 Effectiveness of TPOC in Connecting to Sources of Phase III Funding

Phase II Respondents (Percent)
Very helpful 17.1
Somewhat helpful 27.4
Not very helpful 24.8
Not at all helpful 30.6
Total 100.0
N= 689

SOURCE: 2011 NRC Survey, Question 51.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-17 Ease with Which Principal Investigator Could Contact TPOC

Phase II Respondents (Percent)
Very easy 31.5
Easy 58.2
Hard 8.5
Very hard 1.7
100.0
N= 752

SOURCE: 2011 NRC Survey, Question 52.

Time Available for TPOC to Work on Surveyed Project

Because they have many other responsibilities, TPOCs may not have enough time to work on all of projects in their portfolio. However, in general, survey responses refuted this concern: almost 80 percent of respondents reported sufficient or more than sufficient TPOC time available (see Table 6-19).

TPOCs: Conclusions

Overall, TPOCs are not performing poorly. In general, they are available to SBIR companies and perform many of the key needed functions. However, there is evidence that the TPOCs have a limited impact, particularly in forging connections between their SBIR companies and other nodes in the acquisitions network or with technical resources. This deficiency is clearly one of the drivers behind recent changes at AF, where dedicated staff have been assigned to this function.

The survey revealed a wide range of effectiveness on several measures, suggesting either that TPOCs need better training and guidance or that they are motivated by incentives that are not well aligned with program objectives. The fact that some TPOCs perform a range of functions very well while others do not strongly suggests room for improvement. Finally, given that about one-third of respondents reported a TPOC change during Phase II, which may severely impact outcomes, the components should look into mechanisms to address turnover.

TABLE 6-18 Replacement of TPOC during Award Period

Phase II Respondents (Percent)
Yes 32.3
No 67.7
Total 100.0
N= 750

SOURCE: 2011 NRC Survey, Question 53.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-19 TPOC Time Availability for Surveyed Project

Phase II Respondents (Percent)
More than sufficient 6.9
Sufficient 72.6
Insufficient 20.5
Total 100.0
N= 751

SOURCE: 2011 NRC Survey, Question 54.

ADDRESSING WOMEN AND MINORITIES

The committee sought documentation on outreach to women and minorities from OSB and from the major components. The research team also conducted interviews with agency staff at several DoD components. And in February 2013, the committee hosted a workshop at the National Academy of Sciences on diversity in the SBIR program.23 As well as hearing from other experts and stakeholders, the workshop provided agencies with the opportunity to discuss their efforts in this area, and to identify areas for possible new initiatives. However, the committee was not able to identify any systematic efforts to expand the participation of companies that are majority owned by women or socially and economically disadvantaged groups. In addition, there has not been a concerted effort to attract PIs from these demographic groups. Box 6-4 highlights a variety of perspectives offered by the participants of this meeting to improve the participation of woman and minority-owned small businesses in the SBIR program.

The committee does not support the notion of quotas for demographic groups in an innovation program where merit review drives selection. Moreover, the courts have in recent years rejected mandatory quotas for jobs, for university entrance, and for government contracting.24 Similarly, a 1976 U.S. Supreme Court decision in Craig v. Boren required the application of “intermediate scrutiny” for programs giving preferences by gender.25

The rejection of quotas does not however mean that DoD cannot simply ignore one of the four Congressionally mandated objectives for the SBIR/STTR program: to “foster and encourage participation in innovation and entrepreneurship by socially and economically disadvantaged persons.”26

__________________

23Workshop on “Innovation, Diversity, and Success in the SBIR/STTR Programs” February 7, 2013, The National Academies, Washington, DC.

24Under the 1995 landmark decision, Adarand Constructors, Inc. v. Pena,24 the U.S. Supreme Court required the federal government to apply a high standard of “strict scrutiny” to justify race- and gender-based preference programs.

25Craig v. Boren, 429 U.S. 190, 197 (1976).

26SBA: SBIR Mission and Goals, <http://www.sbir.gov/about/about-sbir>, accessed August 27, 2012. This definition has historically been taken to include women. A detailed SBA definition of

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 6-4

Improving Participation by Women and Minorities in the SBIR Program

To address the question of how SBIR could better address its mandate to encourage the participation of women and minorities, the committee convened a workshop in February 2013 on “Innovation, Diversity, and Success in the SBIR/STTR Programs” at the National Academy of Sciences. Workshop participants discussed a variety of strategies for consideration across the program, many of which are summarized below. Chapter 7 of this report lists the committee’s own recommendations to improve the participation of women and minorities in the Department of Defense (DoD) SBIR program.

  • Participants discussed how SBIR agencies could improve outreach to educate women and minorities on the SBIR opportunity to organizations, including Historically Black Colleges and Universities, women minority advocacy organizations, and professional societies, through workshops, webinars, and social media.
  • Participants discussed how SBIR managers could be incentivized to foster and encourage this key goal of the SBIR program.
  • Participants explored how SBIR agencies might use supplemental agency funds to encourage women and minority participation in their program.
  • Participants urged SBIR agencies to identify and adapt agency best practices, such as National Science Foundation (NSF) programs to encourage women and minorities across the length of the career pathway.
  • Participants suggested that SBIR agencies speed up processing of awards and contracts, given that delays in disbursing funding particularly affect promising though vulnerable firms.
  • Some participants also recommended that SBIR agencies and the Small Business Administration (SBA) develop data and analyze success rates for minority and women applicants, including tracking the number of submissions with successful awards for Phase I and the Phase I to Phase II conversion rate. It was also suggested that the agencies gather and analyze feedback data from women and minority participants and first-time applicants.
  • They called on the DoD and National Aeronautics and Space Administration (NASA) SBIR programs to incentivize their prime contractors to track and encourage women- and minority-owned companies.

__________________

“socially and economically disadvantaged” is available at <http://www.sbda.com/sba_8%28a%29.htm>, accessed August 27, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
  • Some participants suggested that SBA should reassess the definition of women- and minority-owned businesses: For example, is a 51 percent minority and women ownership requirement a barrier to attracting additional investment?
  • Some participants called on SBA to study other models to encourage women and minority entrepreneurship. This includes pre-SBIR programs, such as the Phase Zero programs under way in some states. Other programs identified as worthy of further study included university-based initiatives to encourage women and minority entrepreneurship.
  • Some participants called on SBA to commission a study of the impact of major demographic trends on entrepreneurship.
  • A number of participants highlighted the role that universities can play in incentivizing women and minority professors and students to become entrepreneurs. In this regard, they noted that some universities provide credit toward tenure for professors who commercialize their research results. Some also provide credit toward graduation for students who participate in such commercialization initiatives.
  • The participants noted that universities can also provide training on how to apply for SBIR, including entrepreneurship classes for doctoral students.

More broadly, many participants recognized the need to promote multidisciplinarity and diversity as contributors to innovation. They noted that disciplinary silos in science and engineering and related academic cultures discourage diversity in participation.

DoD data on applications from and awards to woman-owned small businesses (WOSBs) and minority-owned small businesses (MOSBs) are reviewed in detail in Chapter 2. The analysis reveals not only that the numbers are low, but also that the trend shows no clear improvement over time even though the share of women and minorities in the science, technology, engineering, and mathematics (STEM) workforce continues to grow.

Survey Data on Socially or Economically Disadvantaged (SED) Scientists and Engineers in the DoD SBIR Program

Previous discussions of woman and minority participation in the SBIR program focused largely on WOSBs and MOSBs.27 In general, these studies did not address the role of PIs, nor did they disaggregate MOSBs by ethnicity. The current study expands the analysis in both directions.

__________________

27See for example GAO, Small Business Innovation Research: SBA Should Work with Agencies to Improve the Data Available for Program Evaluation, GAO-11-698: Published: Aug 15, 2011. Publicly Released: Sep 14, 2011.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

Socially and Economically Disadvantaged Groups

To the committee’s knowledge, the current survey is the first to probe beneath standard definitions of “socially and economically disadvantaged” (SED). That is, previous SBIR surveys by the NRC and the agencies (and agency data itself) sought to determine whether the company is majority owned by members of socially and economically disadvantaged groups (SEDGs) as defined by SBA.28

Such an analysis is insufficiently granular, because it fails to address important differences within the broad set of SED groups. The current NRC survey addresses this issue by seeking more detailed demographic information from respondents. It builds on Survey 1.0 by addressing the ethnicity of principal investigators (PIs), who often play an important role in the formation of MOSBs and WOSBs.

Socially and Economically Disadvantaged PIs

As with the 2005 Survey, respondents were asked whether the PI for the surveyed project was SED. About 11 percent reported this to be the case (Table 6-20).

The 2011 survey also requested details about the PIs’ ethnicity, according to categories derived from SBA definitions, with the addition of an “other” category to ensure that all respondents who wished to claim SED status could. This detailed question revealed some important differences between SED groups (there were no significant differences between Phase I and Phase II respondents) (see Table 6-21). At least 80 percent of SED Phase I and Phase II PIs were Asian Indian or Asian Pacific in ethnicity. In contrast only 9 percent of PIs were Hispanic, 4 percent were Black American, and 3 percent were Native American.

TABLE 6-20 SED PIs as Percentage of Total

Principal investigator for this SBIR award was from a socially/economically disadvantaged Phase I Respondents group (Percent) Phase II Respondents (Percent)
Yes 10.3 11.1
No 89.7 88.9
Total 100.0 100.0
N= 390 763

SOURCE: 2011 NRC Survey, Question 14B.

__________________

28Different agencies use different terminologies, which change over time. “Minority” is a widely used term, but “socially and economically disadvantaged” is also in use.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-21 Composition of SED PI Grouping, by Ethnicity, as Percentage of SED PIs

SED Group Phase I Respondents (Percent) Phase II Respondents (Percent)
Asian-Indian 47.5 37.6
Asian-Pacific 32.5 43.5
Hispanic 7.5 9.4
Black American 5.0 3.5
Native American 5.0 2.4
Other 2.5 3.5
100.0 100.0
N= 40 85

SOURCE: 2011 NRC Survey, Question 14C.

These data can be placed in the further context of the survey population as a whole. Overall, of the 1,155 DoD respondents to this question, 1 percent reported said that the PI on the surveyed project was Hispanic, 0.4 percent was Black American, and 0.3 percent was Native American. Overall, 89.2 percent were not SED (see Table 6-22).

SED Company Ownership

Turning from the ethnicity of the PIs of the surveyed projects to the ethnicity of the owners of the surveyed companies, about 10 percent of respondents reported that the company was majority owned by SEDs at the time of the award (see Table 6-23).

However, probing more deeply into the ethnic distribution of SED company owners reveals that the distribution is quite similar to that for SED PIs, which suggests that pipeline theory may be valid: that SED PIs may over time become SED owners. In addition, for smaller firms in particular, the owner may

TABLE 6-22 SED PIs at DoD, by Ethnicity, as Percentage of all Respondents

Phase I Respondents (Percent) Phase II Respondents (Percent)
Asian Indian 4.9 4.2
Asian Pacific 3.3 4.8
Hispanic 0.8 1.0
Black American 0.5 0.4
Native American 0.5 0.3
Other 0.3 0.4
All SED 10.3 11.10
N=3 90 765

SOURCE: 2011 NRC Survey, Question 14C.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-23 SED Majority Ownership

Phase I Respondents (Percent) Phase II Respondents (Percent)
Yes 9.3 10.5
No 90.7 89.5
100.0 100.0
N= 388 759

SOURCE: 2011 NRC Survey, Question 19B.

also be the founder and the PI. The percentage distribution of SED ownership by ethnicity is summarized in Table 6-24.

The most important point is clear enough, that is, further disaggregation of the SED category reveals low levels, in particular, of black-owned firms winning SBIR awards. Survey responses identified 5 black-owned SBIR Phase I awards and 1 black-owned Phase II award, for a total of 6 out of 1,155 awards surveyed.

Participation of Women in SBIR

Women have traditionally been viewed as socially and economically disadvantaged in the context of the SBIR program, and expanding opportunities for women has been one of the Congressionally mandated goals for the program since its inception. Both agencies and SBA has focused on the participation of woman-owned companies. However, case studies (e.g. TSI) and other discussions with company executives suggest that being a PI leads to company ownership, so the 2011 survey was revised to capture the extent to which SBIR awards went to female PIs.

TABLE 6-24 SED Company Ownership, by Ethnicity and Phase

Phase I Respondents (Percent) Phase II Respondents (Percent)
Asian Pacific 33.3 41.3
Asian Indian 41.7 36.3
Hispanic 13.9 17.5
Black American 13.9 1.3
Native American 5.6 2.5
Other - 2.5
N= 36 80

NOTE: Columns do not sum to 100 percent because respondents were permitted to select more than one category.
SOURCE: 2011 NRC Survey, Question 19C.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

Female PIs

Two findings clearly emerge from the survey responses (see Table 6-25). First, overall, few female PIs are in the DoD SBIR program. Although it is beyond the scope of this study to make comparisons with the overall populations of scientists and engineers, female PIs received only 7 percent of DoD SBIR awards during the study period.

Second, less than 6 percent of Phase II awards went to female PIs, compared to 9.5 percent of Phase I awards. This indicates that projects with female PIs were about one-third less successful in converting to Phase II than were all PIs.

Woman-owned Businesses

The survey also addressed the extent to which SBIR awards were made to woman-owned businesses. Although the percentage of woman-owned firms in the sample was not large, it was distinctly higher than the percentage of female PIs, particularly for Phase II. In addition, although more Phase I respondents than Phase II respondents reported working at a woman-owned business, the difference was not as great as for female PIs (see Table 6-26).

Outreach to WOSBs and MOSBs at DoD

There is little documentation on efforts to attract more WOSBs and MOSBs to the DoD SBIR program. The annual reports to Congress do not describe efforts, and formal requests for information to the Services resulted in no relevant information sources. Interviews with agency staff revealed that outreach has not been a priority for program managers. That said, there have been some efforts, in particular at Navy. A white paper submitted by Richard McNamara, former leader of Team SUBS,29 is summarized in Box 6-1. After implementation of this outreach program to WOSBs, Team SUBS leadership made additional efforts to reach out to in particular to Hispanic WOSB as well as other WOSBs.

__________________

29Navy defines Team Subs as “an amalgamation of the Program Executive Office, Submarines (PEO SUB), the Deputy Commander, Undersea Warfare (NAVSEA 07) and the Deputy Commander, Undersea Technology (NAVSEA 073). The Team Submarine concept unifies once diverse submarine-related commands and activities into a single ‘submarine-centric’ organization with the goal of eliminating traditional ‘stovepipe’ structures and processes that created impediments and inefficiencies in the submarine research, development, acquisition, and maintenance communities. Team Submarine provides improved communication among the various offices that contribute to the overall success of the United States Submarine Force.” Department of the Navy, Research, Development, and Acquisition web site, <http://acquisition.navy.mil/home/organizations/peos_drpms/peo_subs>, accessed October 10, 2013.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-25 Gender Distribution of Responses, by Phase

Principal Investigator for this SBIR award was a woman Phase I Respondents (Percent) Phase II Respondents (Percent)
Yes 9.5 5.8
No 90.5 94.2
100.0 100.0
N= 388 762

SOURCE: 2011 NRC Survey, Question 14A.

The Team SUBS outreach effort focused on targeted outreach to Historically Black Colleges and Universities (HBCUs) and then later to High Hispanic Enrollment Institutions (HHEs). A total of eight HBCUs were visited (Southern University, Jackson State University, North Carolina A&T University (NCA&T), Howard University, Hampton University, Norfolk State University, Tuskegee University, and Florida A&M University). Later visits focused on HHEs such as Prairie View A&M University in Texas.

At each visit, Navy staff met with deans of engineering and focused on recruiting students for Navy STEM jobs, using STTR and SBIR as mechanisms for introducing students and professors to Navy technology and programs. NAVSEA scholarships were initiated at the 13 schools visited to provide Navy with ongoing presence.

Staff found that, in general, students or faculty did not know about Navy programs, SBIR, or NAVSEA. Navy then proceeded to broker relationships between schools and “proven” SBIR companies to provide solid SBIR/STTR partners. Navy funded Battelle to develop a list of skills and categories across all engineering programs at HBCUs. Navy staff also explored a distributed University Affiliated Research Centers arrangement for all HBCUs, which would make it easier for schools to work with Navy.

The eventual result was that two STTRs were brokered with NCA&T, and a small business emerged nearby that became a successful SBIR company and DoD contractor (3Phoenix). However, although these projects successfully led to Phase II funding, there was no follow-on funding beyond that.

Since the end of the 2000s, there have been minimal efforts to build further outreach programs, even at Navy.

TABLE 6-26 Woman-owned Businesses by Phase

Woman-owned Phase I Respondents (Percent) Phase II Respondents (Percent)
Yes 13.8 10.1
No 86.2 89.9
Total 100.0 100.0
N= 390 761

SOURCE: 2011 NRC Survey Question 19A.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 6-5

Outreach to WOSB at Team SUBS

Efforts to expand outreach to WOSBs began in about 2001. Discussions with the key prime contractors (Electric Boat and Northrop Grumman Newport News) revealed that there were not enough WOSBs to meet contracting goals.

  • Team SUBS outreach conferences for WOSB begin.

o   Fall 2001. First WOSB outreach conference for Team SUBS at Sweet Briar College. The objective was to link WOSBs to acquisitions officers.

o   Fall 2004. Third WOSB outreach conference.

o   2005 onward. WOSB outreach conferences held every other year.

o   Overall, about 200 WOSB introduced to key buyers—Navy labs, prime contractors, support contractors, NAVSEA acquisitions officers, 2001-2005.

  • SBIR program presented as a centerpiece of small business strategy during annual Team SUBS meetings during this period.
  • 2004. Incentive fee introduced into new prime contract for Electric Boat, which acknowledged that it now knew more WOSBs with which to work.

o   Focus on three groups: small businesses, 8(a) businesses, and WOSBs.

o   “Stretch goals” were established over currently subcontract rates (these goals were approximately 35 percent for small business, 4.2 percent for 8(a) businesses, and 3.3 percent for WOSBs).

o   Evaluation occurred in Year 5 of the prime’s contract (2009), after all subcontracts had been placed.

o   Outcomes: 54 percent of contracts by value went to small business, 4.8 percent to 8(a) firms, and 6.6 percent to WOSB. Electric Boat was paid most, but not all, of the incentive fee. The share of subcontracts to woman-owned small businesses doubled during the incentive period.

____________________________

SOURCE: Richard McNamara, “Outreach to WOSB and MOSB in the Navy SBIR program,” White Paper, June 2013.

Conclusions: Women and Socially and Economically Disadvantaged Groups

Aside from the steps described above at Navy, which focused on WOSBs more generally rather than woman-owned SBIR companies or woman-

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

owned technology companies, there is no documentation of any persistent efforts to reach out to women and other disadvantaged groups.

As a result, there is no evidence that DoD is meeting the Congressional goal to “foster and encourage” the participation of these groups in the DoD SBIR program. Indeed, the most recent data suggest that the number of such firms in the program has remained flat for women and has decline for minorities over the past decade (see Chapter 2).

OTHER PI DEMOGRAPHICS

Age Demographics

Other demographic characteristics of the PI population within the DoD SBIR program are of interest. While there is no evidence from DoD beyond the survey results discussed below, other agencies have already focused on this issue. According to Sally Rockey, “In 2010, the average age of NIH principal investigators was ages 53 or 54, and 10 percent of NIH principal investigators were over age 65, a significant increase from 1980 when the average age of NIH principal investigators was 36 or 37.”30 The Committee has accordingly hypothesized that, with the aging of the baby boomers, the age profile of PIs at DoD is likely to shift. Survey respondents were asked about their age at the time of the award. The distribution by age was largely similar for Phase I and Phase II. There were more Phase II respondents aged 40-44 years, and more Phase I respondents aged 65 years or older. About 28 percent of respondents overall were older than 54, and about 22 percent were under 40. Fewer than 10 percent of Phase II PIs were younger than 35 (see Table 6-27).

The limited number of PIs at both ends of the age spectrum suggests a challenge for SBIR programs. On the one hand, breakthrough technologies may predominantly be developed by younger scientists and engineers, so the limited number of awards for younger applicants may indicate over-reliance on prior track record in selecting awardees. On the other hand, changing demographics in the United States indicate that successful research programs will have to engage higher numbers of older scientists and engineers, because they are becoming a larger percentage of the total science and engineering workforce.

Citizenship and Immigration Status

Given the considerable debate in recent years about visas for highly skilled technology workers, and the role of foreign-born entrepreneurs, the committee decided to ask respondents about their citizenship or visa status. The

__________________

30Workshop on “Innovation, Diversity, and Success in the SBIR/STTR Programs” February 7, 2013, The National Academies, Washington, DC.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

TABLE 6-27 Respondents by Age of PI at Time of Award

Phase I Respondents (Percent) Phase II Respondents (Percent)
20-24 0.3 -
25-29 2.1 2.6
30-34 10.0 6.6
35-39 12.8 11.5
40-44 11.0 18.4
45-49 17.2 17.7
50-54 15.9 16.8
55-59 12.6 12.1
60-64 7.9 8.8
65 or older 10.3 5.5
Total 100.0 100.0
N= 390 762
MEAN 49 48
MEDIAN 48 47

SOURCE: 2011 NRC Survey, Question 15.

data show limited differences between Phase I and Phase II respondents. About three-quarters of respondents were U.S.-born US citizens, and a large majority of the remainder were naturalized U.S. citizens. About 5 percent overall were not citizens (see Table 6-28).

TABLE 6-28 Citizenship and Visa Status of Respondents

Immigration status of the PI at the time of the award Phase I Respondents (Percent) Phase II Respondents (Percent)
American-born U.S. citizen 75.3 78.3
Naturalized U.S. citizen 18.3 16.6
U.S. green card 6.2 4.6
H1 visa - 0.4
Other (please specify) 0.3 0.1
Total 100.0 100.0
N= 388 760

SOURCE: 2011 NRC Survey, Question 16.

Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 170
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 171
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 172
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 173
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 174
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 175
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 176
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 177
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 178
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 179
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 180
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 181
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 182
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 183
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 184
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 185
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 186
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 187
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 188
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 189
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 190
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 191
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 192
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 193
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 194
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 195
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 196
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 197
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 198
Suggested Citation:"6 Program Management." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 199
Next: 7 Findings and Recommendations »
SBIR at the Department of Defense Get This Book
×
Buy Paperback | $64.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Created in 1982 through the Small Business Innovation Development Act, the Small Business Innovation Research (SBIR) program remains the nation's single largest innovation program for small business. The SBIR program offers competitive awards to support the development and commercialization of innovative technologies by small private-sector businesses. At the same time, the program provides government agencies with technical and scientific solutions that address their different missions.

SBIR at the Department of Defense considers ways that the Department of Defense SBIR program could work better in addressing the congressional objectives for the SBIR program to stimulate technological innovation, use small businesses to meet federal research and development (R & D) needs, foster and encourage the participation of socially and economically disadvantaged small businesses, and increase the private sector commercialization of innovations derived from federal R&D. An earlier report, An Assessment of the Small Business Innovation Research Program at the Department of Defense, studied how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs. This report builds on the previous one, with a revised survey of SBIR companies. SBIR at the Department of Defense revisits some case studies from the 2009 study and develops new ones, and interviews agency managers and other stakeholders to provide a second snapshot of the program's progress toward achieving its legislative goals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!