National Academies Press: OpenBook

World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization (1996)

Chapter: CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION

« Previous: WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

4

Characteristics and Metrics for a World-Class Army Research, Development, and Engineering Organization

Working from the discussion of pillars in Chapter 3, the committee developed characteristics and descriptions of the metrics associated with them.

CHARACTERISTICS

For the purposes of this study, characteristics are the distinguishing qualities, properties, or features of the pillars. The previous discussion of the five pillars ended with Figure 3-1, which designated 25 important features. These features characterize the pillars, and the committee believes they represent the distinguishing qualities of each pillar. Accordingly, the committee judged these 25 features to be the characteristics most relevant to an Army RD&E organization. A discussion of these characteristics and how they can be assessed appears below. The discussion follows a pillar-by-pillar format, with particular emphasis on determining metrics for evaluating the characteristics of each pillar.

Customer Focus Pillar

Customer focus is directed toward internal customers (e.g., product development teams) and external customers (e.g., soldiers). Both groups of customers can be surveyed to ascertain their satisfaction with the technological solutions and products delivered, the timeliness of delivery, and the quality of technical capabilities and support provided. Customer involvement in setting program objectives and following program progress can also be evaluated.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Although an RDEC must focus on the primary markets it serves, the committee believes that some market diversification (i.e., looking to related markets for technology or products) is proper for any RD&E organization. Indeed, in the private sector world-class RD&E organizations seek to exploit fully the results of their research and product development. Market diversification has merit especially for an Army RD&E organization that has DoD-wide or even broader responsibilities. The extent of market diversification can be determined by examining such things as (1) the penetration and extension of markets for products and technologies and (2) whether these markets lead to expanded or entirely new lines of business. However, the committee notes that market diversification must be considered carefully in the case of Army RDECs because they exist primarily to support their Army missions. Also, RDECs rely on government funding, which is usually authorized to satisfy specific government needs rather than to diversify markets served by individual organizations.

Measuring customer satisfaction and customer involvement in research, development, and engineering, and the nature and appropriateness of market diversification, indicates how well the RDEC is connected with and focused on the long-term and short-term needs of the various customers the RDEC serves.1

Resources and Capabilities Pillar

Resources and capabilities can be evaluated in terms of personnel (i.e., human resources); facilities and infrastructure (i.e., physical resources); budget (i.e., financial resources); RD&E capabilities, skills, and talents (i.e., intellectual resources); the use of external resources; important technologies for each directorate; applied information technology; and organizational climate. Internal (e.g., staff and management) and external (e.g., peer, higher headquarters, and customer) reviews can be conducted periodically to assess resources and capabilities. These reviews may include analyses of the core technical programs, evaluations of employee morale and the research climate,

1  

Other indicators can be used to assess customer focus (e.g., an organization 's success at anticipating unstated customer needs or how well an organization “hears” the voice of the customer in determining overall direction). However, the committee decided that, for an Army RDEC, it is preferable to concentrate on satisfaction, involvement, and diversification.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

and assessments of the ability and effectiveness of program managers to acquire technology from outside vendors (i.e., reach “make versus buy” decisions). Measuring the quality and quantity of the human, physical, and financial resources and the core capabilities of the RDEC gives an indication of the ability and power of the RDEC to achieve world-class results. A positive organizational climate is most often correlated with high productivity (Miller et al., 1996).

Strategic Vision Pillar

The strategic vision must be shared with (i.e., communicated to and discussed with) and understood by staff and stakeholders alike. The vision must then be translated into action. How well this is done depends on the quality of the leadership of the RDEC. Reviews to assess strategic vision should include assessing RDEC leadership. Internal and external (e.g., peer) reviews can determine if the strategic vision and the mission are aligned. They can also determine if anticipatory strategic planning is sufficient to develop future Army and joint service products rapidly. To assess stakeholder buy-in, important classes of stakeholders must first be identified; then the extent to which buy-in by particular stakeholders is critical and the degree to which buy-in is obtained can be assessed. Measuring the quality of the strategic vision of the RDEC (i.e., the extent to which the vision and mission are aligned, to which planning is anticipatory, and to which stakeholders “live” the vision) will give a reading of the enduring capability of the RDEC to plan and achieve world-class results.

Value Creation Pillar

Value creation is often a perception. Perceived values are based on comparisons of the benefits (or lack thereof) of previous products with current products or of the properties of products and costs. Another perception is whether the right products are being delivered to the right place at the right time. Reviews of the breadth of RD &E (i.e., the portfolio of programs), the performance of products and the benefits of services, cycle time and responsiveness, and the value of work in progress are all important to assessing value creation (Miller et al., 1996). Reviews can be both internal and external evaluations of programs in progress, products developed, or services delivered. Measuring the

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

extent to which the RD&E organization produces outstanding, meaningful results (i.e., creates value for customers) yields an understanding of the present and potential impact of the organization.

Quality Focus Pillar

Several important characteristics are associated with quality focus. First is a capacity for scientific, technological, and engineering breakthroughs. This characteristic can be assessed, in part, by reviewing past performance (e.g., how many breakthroughs have already occurred). Next is the ability to improve continuously, which can be assessed by reviewing specific efforts to improve processes and products. Commitment to quality products and services must be assessed at all levels, from topmost management to the lowest working level. To improve the quality of work, work processes must first be understood, defined, and (to a degree) structured. Structured processes (i.e., working in a disciplined and organized fashion) can be assessed by examining processes and results. Reviews can determine the ability of staff members and the organization as a whole to learn, acquire knowledge (and capabilities), and use this knowledge to achieve outstanding results. Finally, the quality of research can be assessed by expert reviews on several levels (e.g., to determine whether high standards of technical excellence are being maintained). Measurements of all these characteristics can give an overall assessment of the focus on quality in an Army RD&E organization.

METRICS

For the purposes of this study, metrics are defined as standards for measuring the characteristics of each pillar. Before applying metrics in a particular situation, it is necessary to understand some aspects of metrics in general. This understanding is revealed by the answers to several questions.

Who, What, Why, When, and How

Metrics can be used in many ways for evaluating an RD&E operation (IRI, 1996). Long lists of metrics have been developed, but selecting

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

the most pertinent metrics is important. Critical questions can be asked about using metrics, the answers to which are applicable to Army RDECs.

Who develops the metrics?

Metrics can be established by the involved group (e.g., an RDEC or its directorates) or by others interested in the performance of the group. The metrics must be understood by those making assessments and by those being assessed.

What specific metrics should be developed?

Different sets of metrics are meaningful to different groups. Metrics must be useful for the organization; specific metrics will drive the behavior and actions of people within the organization. For an RD&E organization, metrics should foster improvement and be related to the vision and mission of the organization.

When selecting metrics one must keep in mind that RD&E efforts are part of a system (Brown and Svensen, 1988). Figure 4-1 shows input into the system, which comes from various sources (e.g., personnel, dollars, equipment, scientific knowledge) and moves through RD&E processes to produce output (e.g., reports, patents, concepts). Product output is converted by receivers into outcome (e.g., new products, services, cost savings, benefits for soldiers). 2

Metrics can be developed to measure (1) the quality of input, (2) the operation of RD&E processes, (3) the quality of output, and (4) the value of outcome. Also, metrics can be developed that relate these four items to the past, present, and future. For example, variations on one measure of an RD&E organization's output might be the number of patents used in the last five years (past), number of patents filed (present), and the number of patent disclosures anticipated during the next five years (future).

2  

The portrayal of RD&E as a system is not meant to exclude the myriad of interactions with external stakeholders, especially customers. The diagram is useful for isolating several key RD&E processes. However, from a larger perspective (e.g., an entire business), the RD&E system is only a subsystem.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

FIGURE 4-1 Research, development, and engineering as a system.

Why use metrics?

Metrics can help an organization assess and determine business or technical objectives, encourage changes, and serve as mechanisms for planning, screening (e.g., setting priorities for improvement), and managing RD&E programs. Metrics can also measure contributions from management, research scientists and engineers, and support staff toward developing, producing, and delivering products.

When should metrics not be used?

The wrong metrics can be harmful. Some metrics may actually limit performance or lead to inappropriate behavior, actions, or results (e.g., metrics based on false cause-effect relationships or wrong work-process models).

When should metrics be developed?

Ideally, metrics should be developed as part of the process of setting objectives or part of the evaluation of how well objectives have been met.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

How are metrics developed?

Metrics can be developed collectively or individually, and they can be tailored to match the unique environment of the group or organization being assessed.

Referring to metrics described by the Industrial Research Institute (IRI, 1996) and others (Roussel et al., 1991; Davidson and Prudent, 1996), the committee developed a set of metrics to assess Army RD &E organizations like the Natick RDEC. Four metrics were developed for each of the 25 characteristics identified at the beginning of this chapter.

Of paramount concern during the development of the metrics was the committee's desire to emphasize the importance of demonstrated commitment by the organization's senior leadership (and, where applicable, the next level of command) to attaining and maintaining world-class performance throughout the RD&E organization. Without this commitment, attaining world-class performance will be impossible.

Applying Metrics

The 100 metrics described below can be used to monitor improvements and assess the Natick RDEC in terms of world-class performance. The metrics can also be used by Natick RDEC or other RDEC personnel for self-evaluation or by higher-level Army commands for evaluating other RD&E organizations.

In other situations metrics are used to provide numerical measurements, such as the number of patents written per year or the number of Ph.D.s on staff. Although these measures clearly are relevant to the performance of an RDEC, the committee does not believe that numbers, by themselves, accurately describe the many facets of RDEC performance.3 Accordingly, the metrics chosen for this assessment are based on qualitative descriptors for four levels of performance (Adler

3  

“Numerical indexes are easily understood and readily compared.... However, there are also some serious drawbacks associated with quantitative measures that counterbalance their virtues....They don't work well in professional groups, such as R&D organizations, where much of the work is characterized by uncertainty and variability…” (Brown and Gobeli, 1992).

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

et al., 1992).4 These levels are poor, adequate, good, and excellent.5 The committee believes for an organization to be deemed world-class, performance must be predominantly excellent.

The committee also considered the category of best-in-class, which is a level of performance beyond excellent. Best-in-class is not included in the following metrics because descriptors for this category apply to unique situations (e.g., an organization or process that is, indeed, the very best) and might not be applicable in a situation where there are no comparisons.

Table 4-1, Table 4-5 summarize, pillar by pillar, the committee's metrics and the four levels of performance for each characteristic. These tables are not necessarily all inclusive; additional tailored characteristics and metrics may be necessary for some evaluations. The descriptors should be looked upon as conceptual guides that can be modified for specific situations.

The results of assessments can be recorded as illustrated in Table 4-6. Overall assessments for each pillar would be arrived at on the basis of judgments of the assessments for each characteristic in that pillar. The committee believes that good or excellent performance in each characteristic, and excellent overall performance in all five pillars, are necessary for an organization to be judged world-class.

The results might also be presented in graph form in typical “spider diagrams” (see Figure 4-2). Spider diagrams may have several uses. For example, Figure 4-2 could be a convenient summary showing if an RDEC approaches or achieves world-class performance on the basis of the pillars alone. A more complicated diagram could include radials for each of the 25 characteristics. Spider diagrams for each directorate could provide visual comparisons. In the case of multiple assessments, a spider diagram for the Natick RDEC could be compared with diagrams for other RDECs.

4  

Many recent metrics (e.g., the Industrial Research Institute's Technology Value Program) that began with the Adler article now use the four-stage model; hence the committee adopted this model to be more consistent with the current literature and the practice of several companies.

5  

The committee also considered using the terms Stage 1, etc. to designate levels of performance. However, the committee was concerned that using numbered stages might lead to a “numbers game” and an inflexible scoring system; hence the committee decided to use words. The words do have some negative and positive connotations, which are intended.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

FIGURE 4-2 Spider diagram.

It should be noted that the committee has implicitly given equal weight to all five pillars. Under some circumstances, it may be appropriate to assign greater weight to one pillar or another.

Finally, the committee observes that application of the metrics in this report need not be limited to organizations striving to attain world-class performance. For example, if the metrics are used for self-assessment or self-understanding, they should be used in the context of an organization's role, responsibilities, and goals. If the organization's goals do not include world-class performance, then the organization could choose to use the metrics and approach the assessment in a different way. In some cases an organization could decide to use the metrics as a guide for self-assessment and develop its own definitions of poor, adequate, good, or excellent. Also, an organization might wish to focus on some characteristics or pillars more than others. This adaptation process in itself can be important for self-improvement.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

REFERENCES

Adler, Paul, William McDonald, and Fred MacDonald. 1992. Strategic Management of Technical Functions. Sloan Management Review. vol. 33 (2): 19–37.

Brown, Warren and David Gobeli. 1992. Observations on the Measurements of R&D Productivity: A Case Study. IEEE Transactions on Engineering Management. 39 (4): 325–331.

Brown, Mark and Raynold Svenson. 1988. Measuring R&D Productivity. Research-Technology Management. 31 (4): 67–71.

Davidson, Jeffrey and Ann Lorette Prudent. 1996. Quality Deployment in R&D Organizations. Research-Technology Management. 39 (1): 49–55.

IRI. (Industrial Research Institute). 1996. Measuring the Effectiveness of R&D: A set of 50+ metrics available on a searchable disk. Available from IRI, Suite 1100, 1550 M Street, N.W., Washington, D.C., 20005-1708. (Phone: 202-296-8811).

Miller, Joseph, Parry Norling, and John Collette. 1996. Research/Technology Management: Leading, Managing, and Getting Results . Kirk-Othmer Encyclopedia ofChemical Technology. Volume R.In press.

Roussel, Philip, Kamal Saad, and Tamara Erickson. 1991. Third Generation R&D. Cambridge, Mass.: Harvard Business School Press.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Tables 4-1 through 4-6

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-1 Metrics of the Customer Focus Pillar

Characteristics

Performance Level

Metrics

Customer Satisfaction

Poor

Less than satisfied or dissatisfied with

  1. strategy used to develop the product or service, appropriateness of the technological solutions, fulfillment of operational capability requirements

  2. technical capability, quality, and performance of the service or product

  3. product cycle time and delivery time of the first equipped unit

  4. technical support for fielded products developed at the RD&E organization

  5. technical capabilities of the product or service of the RD&E organization

 

Adequate

Satisfied with all of a–e (met expectations)

 

Good

Very satisfied with a–e (exceeded expectations)

 

Excellent

Delighted with a–e (beyond normal expectations)

Customer Involvement

Poor

No consideration was given to involving either internal or external customers in program planning, evaluation, or early “results” (prototype) testing.

 

Adequate

Internal or external customers are at times consulted on various aspects of research programs or are involved primarily in program reviews.

 

Good

Internal or external customers are from time to time involved in setting program objectives and following progress; there are opportunities for customer feedback.

 

Excellent

Customers feel completely involved, almost like partners in the effort; they feel they can and do have a major impact in the life-cycle development of the product or service.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Market Diversification

Poor

Although diversification is addressed in strategic and business plans, senior management has not effectively broadened the customer base; products are developed only for the Army; few joint service RD& E programs are in place.

 

Adequate

RD&E programs result in products for the Army and the other uniformed services; the organization provides products to other federal agencies; some of the budget is devoted to developing partnerships with industry and academia.

 

Good

The organization is assigned DoD lead on joint programs; a significant amount of the budget is devoted to partnerships with industry and academia; research partnerships yield products that fulfill military needs and fill a void in the needs of other federal agencies.

 

Excellent

As a center of excellence, the organization's products serve a wide range of customers, including DoD, other U.S. government organizations, and global allies of the United States; much technology is transferred between the organization and the private sector; industrial and academic partnerships result in the rapid transfer of cutting-edge technology between the organization and its partners; high-quality products are developed, manufactured, and distributed to global customers.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-2 Metrics of the Resources and Capabilities Pillar

Characteristics

Performance Level

Metrics

Personnel Quality

Poor

Work is below standard throughout the organization; there are inadequate technical skills; program planning and management are poor.

 

Adequate

Work meets expectations; work force has adequate skills to get results in a timely manner; opportunities to improve and upgrade skills are minimal; few resources are programmed for improving technical skills.

 

Good

Work usually exceeds expectations; newly hired employees bring critical new skills and capabilities; present RD&E personnel devote at least a small percentage of their work time to upgrading or acquiring skills, and this training is reflected in annual performance appraisals; the Army gives special recognition to RD&E personnel; personnel are well connected with the scientific and technical community outside the organization.

 

Excellent

Work consistently exceeds expectations (of those with major interests in the work of the organization); new skills and capabilities are regularly introduced into the organization; newly hired personnel bring new, state-of-the-art methods into the organization; personnel are encouraged to devote a significant amount of their work week to improving and acquiring technical skills; personnel are recognized for their accomplishments by individuals and organizations outside the Army; career structures support the development of technologists in a wide range of needed disciplines; personnel are noted for effective use of both external and internal resources.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Budget

Poor

Research and program support budgets are constrained; research programs are consistently underfunded; out-year budget projections are flat or decrease; mid-year budget cuts are routine; programs are abandoned, with resulting inefficiencies.

 

Adequate

Although budgets are at the recommended levels, major research programs are constantly in jeopardy because of uncertainties about year-to-year funding; no new major construction or programs are funded even though budgets finally prove to be adequate for maintaining ongoing programs.

 

Good

The organization consistently finds ways to get more done with less; resources are leveraged with other government agencies; the organization periodically takes the lead in DoD-wide or similar programs; collaborative programs with industry and academic groups are cultivated; some funding is provided to support new research initiatives, acquire new equipment, and construct or renovate laboratory facilities.

 

Excellent

The outstanding work of the organization is recognized by prompt funding at desired levels; the organization is asked to accelerate RD&E programs and initiate new missions when additional funding is available; program managers obtain the absolute best value with their budget; resources leveraged with other organizations and agencies are recognized as force multipliers; the organization maintains a backlog of high-quality yet-to-be-funded projects.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

RD&E Capabilities, Skills, Talents

Poor

Technical skills, capabilities, and talents are inadequate to support current and future customer requirements; few new techniques and skills are acquired via new hires or continuing education and retraining of personnel; personnel cannot fully operate, maintain, or utilize available equipment; continuing education is not promoted, encouraged, or funded.

 

Adequate

Plans are developed and funding is provided for maintaining the present core capabilities for the future; personnel are trained to operate and maintain equipment and use equipment as specified by the manufacturer; personnel skills are recognized as current and competent for their technical specialties.

 

Good

The organization possesses the skills and talents to fulfill customer requirements for the foreseeable future; new and innovative techniques, skills, and processes are incorporated into the RD&E processes; newly acquired skills result in improved product engineering, manufacturing, or performance; new personnel are recruited to bring state-of-the-art techniques into the organization; personnel are encouraged to participate in formal continuing education programs; members of the research staff are encouraged to participate in professional societies, serve on external committees, etc.; program managers recognize new skills that will benefit their programs, and they plan for the acquisition of these skills and talents.

 

Excellent

The research and support staffs are recognized as possessing superb technical and administrative skills and talents; many members of the support staff are recognized as artisans of their trade; research personnel incorporate state-of-the-art techniques into their research and develop pioneering methods of their own; a clearly articulated plan describes how needs and voids in core capabilities are identified and filled; new capabilities that must be developed are also addressed and acted upon; a growing inventory of skills is maintained.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Use of External Resources

Poor

Work is contracted outside the organization on an ad hoc basis with little or no planning; contract managers do not ensure that statements of work are fulfilled on time or on budget; products and services provided by contractors and partners contribute incrementally to the organization's mission.

 

Adequate

Partnerships are developed with a wide range of groups to enable work to be done outside the organization; work done by others is contracted primarily based on the other party's willingness to do the work; products and services obtained from external sources fulfill the statement of work; products complement internal research programs.

 

Good

The organization is recognized as a “smart buyer” of services and work of other parties; personnel appreciate the quality of the work that is contracted; the extent of leverage (i.e., the ratio of the cost to do the work at the organization to the contracted cost) is appreciated; external research programs enhance internal programs and result in leap-ahead technology.

 

Excellent

Partnerships and contracts with organizations recognized as the best in their field complement RD&E programs and result in leap-ahead (and occasional breakthrough) technological advances; skills and abilities of the external organization cannot be duplicated in the organization in a cost-effective manner; the value of partnerships is widely recognized inside and outside the RD&E organization.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Important Technologies

Poor

There are no systematic programs or processes for introducing, managing, or assessing research technologies in the research program.

 

Adequate

Base technologies being developed or used in the research program are necessary for fulfilling technological needs but offer little differentiation in product performance from other alternatives; important technologies are recognized, developed, and used, but technology development is not advanced.

 

Good

Pacing technologies are being developed or used in the research program; these technologies have the potential to change significantly the nature of the research program, but they are not yet embodied in products; incorporation of pacing technologies results in leap-ahead developments.

 

Excellent

RD&E programs are anticipatory; development and incorporation of new technology to support RD&E and product development are planned and adequately funded; new areas of research and technology are appreciated, and researchers understand the implications of particular research programs; new scientific discoveries are frequently translated into pacing technologies within the organization.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Organizational Climate

Poor

The work environment is acknowledged by management and staff to be poor; personnel are preoccupied with furloughs, early retirement, and downsizing initiatives; personnel equate re-engineering to organizational instability; initiating risky programs is discouraged; management punishes failure by withholding resources.

 

Adequate

The work environment is perceived as professional and collegial; personnel enjoy their work and say it is meaningful; responsibilities are clear, and teamwork and collaborative efforts are evident; managers tolerate innovation and occasionally empower their staffs, teams, and groups; personnel are recognized for their contributions; although anxious about reorganization and downsizing, individuals feel relatively secure about their jobs.

 

Good

Work and organizational climate is considered good; bold and innovative thinking is encouraged and rewarded; research personnel are fully empowered to set goals and pursue original and innovative solutions, but they do not fear failure; the organization is recognized as possessing a “can do” attitude.

 

Excellent

Management and staff perceive the organizational climate as excellent; there is clarity of purpose and vision; the staff is secure; no hint of fear is present, and rewards and recognition motivate individuals and teams to make excellent contributions; management encourages the development of new work environments that result in increased productivity.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Information Technology

Poor

Computer hardware and software are not available at every work station; software and hardware are two generations or more out of date; personnel cannot communicate electronically or transfer data internally or externally; personnel are poorly trained and hesitant to learn new applications; funding for information technology and user training is inadequate.

 

Adequate

Information technology is used as a tool by research and support personnel, and it increases productivity and ultimately decreases the organization's overhead; acquisition of new hardware and software is adequately funded; training and technical support are available; personnel are comfortable with the available technology and are electronically connected internally and externally.

 

Good

An information technology strategy guides program direction; research, support, and administrative systems are integrated; information technology enhances the effectiveness of the RD&E allowing work to be done in entirely new ways; information technology is credited with recent advances in research programs; hardware and software are state-of-the-art; technical support is abundant; the staff is educated in the use and application of the technology.

 

Excellent

Information technology enables rethinking how RD&E is done, and technical breakthroughs, previously thought of as being impossible, are within reach; the products include information-technology components.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Facilities and Infrastructure

Poor

Facilities and equipment are inadequate, poorly maintained, and out-of-date; no new investments in equipment and facilities are forecast;. preventive maintenance is seldom performed; safety and regulatory compliance are rarely addressed.

 

Adequate

Facilities are judged adequate to meet the needs of the organization; there is a schedule for periodic maintenance and upgrading of equipment; safety and regulatory compliance policies are in place, but audits, inspections, and training are limited.

 

Good

Research and support facilities are clean, spacious, and comfortable; facilities are environmentally controlled year round; equipment is upgraded or replaced routinely; preventive maintenance and service contracts are well funded; relatively new technical capabilities are acquired, and user training is provided; there is evidence that safety and regulatory compliance are important (e.g., statistics are maintained, periodic inspections are made versus appropriate standards, and training is emphasized).

 

Excellent

Facilities and equipment are exceptional; there is timely access to equipment and facilities, which aid personnel in many unexpected ways (e.g., the latest technologies allow them to look at problems in new ways; specialized analytical equipment opens new horizons; there is sufficient equipment to meet user needs); critical programs are supported with state-of-the-art equipment; there is pride in the installation's records in safety and regulatory compliance, ample resources are devoted to inspections and training, and employees continually strive for better safety and regulatory compliance.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-3 Metrics of the Strategic Vision Pillar

Characteristics

Performance Level

Metrics

Alignment of Vision and Mission

Poor

Vision and mission statements are not articulated well, nor are they linked; senior management has difficulty communicating vision and mission statements through command briefings, annual plans, or business plans to staff members and customers.

 

Adequate

Vision and mission statements are articulated and understood by most employees; a research strategy is developed using these statements as a guide; research programs, resources, and management support are aligned, in general, with the research strategy.

 

Good

The organization's strategic vision is inspiring, and the vision and mission are in harmony with each other; the vision and mission provide a “guide to action” for all programs; management support and resources are aligned with the research strategy.

 

Excellent

The strategic vision and management's translation of this vision into a research strategy yields superior products and services; the alignment of resources with the research strategy is readily apparent.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Anticipatory Strategic Planning

Poor

No strategic planning process is implemented, or the strategic plan is ineffective.

 

Adequate

A strategic planning process is in place, and business plans and annual plans are implemented; senior management enlists research and support staff assistance to draft and implement the strategic plan through the business and annual plans; the plans are communicated to the staff.

 

Good

A robust planning process is in place, with broad involvement across the organization; the resulting planning document is used to measure progress throughout the year; contingency or alternative plans are developed to accommodate rapid changes in customer needs, the environment, or resources.

 

Excellent

Plans for human resources, information technology facilities, budget, and travel are fully integrated into strategic plans; the planning horizon for the strategic plans is sufficient to anticipate major Army and joint service needs; multiple examples demonstrate a high degree of flexibility within the organization, which has reacted rapidly to either major opportunities or critical customer needs.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Stakeholder Buy-In

Poor

The strategic vision and research plan either have not been communicated to the RD&E stakeholders or have not been articulated well and are misunderstood; stakeholder response to the vision and research plan is either negative or indifferent.

 

Adequate

A strategic vision is spelled out and understood by most stakeholders; the vision makes all major initiatives readily understandable.

 

Good

The strategic vision “speaks” to all stakeholders even if they have not been involved in creating it; customers and disinterested parties understand the research plan and advocate providing adequate resources to implement the plan.

 

Excellent

The strategic vision is so clearly articulated that stakeholders lobby Army and DoD planners to implement the research plan fully; stakeholder support for the organization's vision and the research plan are so strong that resources are reprogrammed from other accounts to implement the vision.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Leadership

Poor

Commitment of the senior leadership to the strategic vision or research plan is poorly communicated to the staff; administrative and product development managers are not involved in planning the direction of future research or developing the business plan; personnel are suspicious or do not trust the organization's leadership; stakeholders view the senior leadership as ineffectual and reactive.

 

Adequate

The strategic vision and research plan are understood by the staff; resources (i.e., time, personnel, and dollars) are aligned to meet these plans; the staff trusts senior leadership and is receptive to new ideas and re-engineering opportunities.

 

Good

Management and staff co-develop plans that are understood and embraced by staff and stakeholders alike; ideas flow freely and in both directions between management and staff.

 

Excellent

The leadership has created an air of excitement and commitment throughout the entire laboratory; bold and creative ideas are encouraged and funded; RD&E successes are rapidly exploited, and ideas are rewarded; failure is considered an opportunity to learn.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-4 Metrics of the Value Creation Pillar

Characteristics

Performance Level

Metrics

Proper Portfolio

Poor

Products are developed that do not meet customer needs; products have poor customer acceptance; customers perceive that commercial alternatives are cheaper, perform better, and are more durable.

 

Adequate

An analytical process to examine the product portfolio is used to design and field products that have greater value and soldier acceptance; results of the analytical process lead to modifications in product design; major changes may be made after fielding the initial product.

 

Good

Portfolio analyses of a program are an integral part of the strategic planning process; there is broad and active customer involvement in the portfolio analysis; programs yield products that have significant customer acceptance, meet or exceed customer requirements, and demonstrate increased value compared to current products or commercial alternatives; minor changes in product design occur after initial fielding.

 

Excellent

Portfolio analyses result in RD&E processes that yield products and services with excellent value, performance, and customer acceptance.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Product Performance

Poor

Products do not meet customer requirements (e.g., in terms of weight, volume, function, durability, or maintainability); customers complain that product performance does not meet the developer's claims; products are not suitable for use in certain locations or environmental extremes.

 

Adequate

Products meet customer requirements, needs, and expectations.

 

Good

Products fully meet or exceed customer requirements; products are perceived as better than the ones they replace.

 

Excellent

Products not only exceed customer expectations, but product performance includes some pleasant, unexpected surprises (e.g., reduced maintenance requirements, longer shelf life, longer mean time to failure, resource savings).

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Cycle Time and Responsiveness

Poor

Cycle time for project completion is longer than anticipated; milestones are routinely missed; program delays result in increased end-item cost; research programs do not anticipate customer needs; management and staff are not flexible to modifications of product requirements.

 

Adequate

Elapsed time from project initiation to project completion is measured and can be reliably forecasted; research programs are described as being on-time and on-budget.

 

Good

RD&E programs are initiated and completed significantly faster than similar government or commercial programs; research staff is responsive to “quick fixes” for troops, and numerous examples are readily available for major products; senior management ensures that adequate resources are reprogrammed to fulfill quick-fix requests.

 

Excellent

RD&E programs are initiated and completed substantially (e.g., one third) quicker than similar government or commercial programs; innovative processes and technical solutions reduce typical quick-fix response times by nearly half; the staff monitors foreign and domestic industrial and academic research for solutions to new and unanticipated technical problems; commanders directly and indirectly express gratitude for responsive quick fixes.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Value of Work in Progress

Poor

No evaluations of historical RD&E programs are available for comparison to current programs; no methodology is in place to assess current RD&E programs; customer perception of prior RD&E programs is predominantly critical and negative, and little or no value is placed upon the current programs by the customers.

 

Adequate

A database on select historical RD&E programs and all current programs is available; current RD&E programs are vividly described, and these descriptions are used during peer-review discussions to justify programs and prioritize personnel and budget requests; customer perception of prior RD& E programs is generally positive; customer perception of current RD&E programs is positive (i.e., the products and services will generally meet user requirements and be delivered on time and on budget).

 

Good

A database is maintained on all past major projects (e.g., for the last decade) and their primary and secondary impacts; the database is used for comparison with current RD&E programs; leadership creates a scale to rate continuously the potential value of current programs compared with previous programs and show improvements; customers rate RD&E programs as very good (i.e., products are expected to fully meet or exceed customer requirements; products are perceived as likely to be better than the ones they replace).

 

Excellent

A complete historical database and evaluation methodology are used to demonstrate the value of the organization's products and services; data are used to justify and defend program expenditures; customers rate products and services as excellent (e.g., product performance exceeds customer expectations); product performance exceeds anything projected to be available from domestic and foreign sources for at least several years.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-5 Metrics of the Quality Focus Pillar

Characteristics

Performance Level

Metrics

Capacity for Breakthroughs

Poor

RD&E programs are routine and unimaginative; there is no evidence of imaginative or innovative solutions being applied to RD&E tasks; resources are directed to meeting specific customer requirements only.

 

Adequate

RD&E programs are characterized by steady but incremental improvement; several innovative solutions can be pointed out; minimal funding is available for programs that anticipate future military requirements.

 

Good

Although most programs are characterized by incremental improvements in technology, the organization has demonstrated several leap-ahead improvements; the organization encourages and funds opportunities to seek truly innovative, moderate-risk solutions.

 

Excellent

Unexpected innovation based on breakthroughs in technology occur fairly regularly among internal and external (cooperative) RD&E programs; moderate- and high-risk research that offers high return receives stable funding; numerous examples of breakthrough research are cited from the previous five to ten years.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Continuous Improvement

Poor

There is no tangible evidence of senior management commitment to continuous improvement; the need and ability to focus on continuous improvement are recognized, but not funded; products and services show incremental changes; innovations are not rewarded; solutions from industry and academia are discounted as “not invented here.”

 

Adequate

Quality of the work is discussed and several measures of quality are used routinely; innovative solutions are encouraged; staff members frequently make suggestions for improvement; several changes are made (and documented) each month for improving the work and the output of the organization.

 

Good

The organization takes steps to improve work processes and RD&E results significantly; quality audits are performed periodically by internal and external review groups; numerous improvements can be pointed out; productivity is an important topic of discussion; report cards are issued annually by senior leadership; senior managers have the resources to enact recommendations.

 

Excellent

Greater productivity, enhanced research and product quality, improved customer involvement and satisfaction, and continuing education of the work force are areas of primary interest to senior management; the concepts of continuous improvement and excellent product value are embedded in the goals of each RD&E and support function; there is a systematic analysis of research and support processes to eliminate non-value-added activities; research personnel are renowned for finding innovative solutions to technically difficult problems.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Commitment to Quality

Poor

Management espouses a commitment to quality, but no formal process to review and evaluate quality is in place; some quality-related results are managed by exception; the quality of products and services varies between RD&E units in the organization.

 

Adequate

Management is investing resources for total quality training and implementation; the variability of products and services is being measured and tracked; personnel are aware of the importance of quality.

 

Good

Total quality implementation is a major goal in the organization's strategic plans; a framework and methodology for measuring and assessing total quality is in place; measurable objectives for work-process improvement are established; there are methods (e.g., statistical process controls) to improve effectiveness and product quality with existing resources.

 

Excellent

The commitment to total quality is inherent and pervasive throughout the organization; the focus of all measurements is on optimizing the RD&E processes to deliver value; frameworks, such as ISO 9000/2 (international quality standards), Baldrige criteria, or locally developed systems, are used for assessment; recommendations to improve quality are immediately funded and implemented.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Structured Processes

Poor

Work processes and procedures are understood and milestones are established, but there is no system of internal or external review; project management results in products or services that are delivered late and over budget; delays result in termination of projects; disciplined approaches to defining problems and the scientific method are rarely used.

 

Adequate

Work processes and procedures are monitored; project costs and milestones are closely tracked; processes are established to improve quality incrementally, contain or reduce RD&E cost, and reduce product cycle time; disciplined approaches and the scientific method are used most of the time.

 

Good

Program managers are flexible and adaptive; senior leadership and staff are receptive to innovative ideas for improving work processes and procedures; product quality and customer focus mean continuous improvement; disciplined approaches and the scientific method are used consistently.

 

Excellent

The senior leadership strives to identify and incorporate best business practices into the organization; processes are considered flexible and not overly restrictive, prescriptive, or bureaucratic; management is focused on achieving superior performance and product quality; emphasis on cross-project management ensures timeliness and the proper allocation of resources; disciplined approaches to problem solving include an extensive network linked to Army technological resources worldwide; the scientific method is strictly followed.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Learning Environment

Poor

Senior leadership is characterized as reactive; little if any learning takes place on an organizational basis; some managers and staff learn from mistakes.

 

Adequate

Senior leadership recognizes and communicates the importance of organizational learning; management and staff learn from mistakes and from others; personnel are well networked both inside and outside the organization; teams on one project teach teams assigned to other projects; new skills and techniques are acquired through new hires and continuing professional education.

 

Good

Organizational learning is characterized as adaptive; the organizational climate is conducive to learning; personnel are rewarded and encouraged for taking risks and entrepreneurial initiatives despite occasional mistakes; personnel learn from others and by doing; management experiments with new organizational concepts to discover new ways of doing things.

 

Excellent

Organizational learning is adaptive and anticipatory; research and technical capabilities continually expand, and management anticipates change; traditional and innovative methodologies are used to measure and evaluate organizational learning.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

Quality of Research

Poor

Research and technology programs are not generally aligned with customer requirements and needs; records of research methodology and results are poor; although recorded in technical reports, data are not published in peer-reviewed journals or cited by other scientists in academia or industry; research results cannot be replicated by scientists and engineers outside the organization.

 

Adequate

Research and technology programs are aligned with customer requirements and needs; research methodology and results are peer-reviewed and published as both technical reports and journal articles; the research staff is invited to participate in scientific meetings and workshops; research results are easily replicated by other laboratories.

 

Good

The research and technology programs are recognized by peers as being of very high caliber; several programs are among the best in the federal government and are described as innovative and original; some patents are awarded.

 

Excellent

The quality of the research and technology programs is considered to be among the best in the world; basic research not only fulfills customer needs, but also anticipates future requirements, thus reducing cycle time for new products; research and technology programs are innovative and state-of-the-art; new procedures, processes, and materials are developed by personnel; numerous patents are issued for RD& E innovations.

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×

TABLE 4-6 Organizational Assessment

 

Assessment

 

Component

Poor

Adequate

Good

Excellent

Customer Focus Pillar

       

Customer Satisfaction

____

____

____

____

Customer Involvement

____

____

____

____

Market Diversification

____

____

____

____

Pillar Assessment:

____

____

____

____

Resources and Capabilities Pillar

       

Personnel Quality

____

____

____

____

Budget

____

____

____

____

RD&E Capabilities, Skills, Talents

____

____

____

____

Use of External Resources

____

____

____

____

Important Technologies

____

____

____

____

Organizational Climate

____

____

____

____

Information Technology

____

____

____

____

Facilities and Infrastructure

____

____

____

____

Pillar Assessment:

____

____

____

____

Strategic Vision Pillar

       

Alignment of Vision and Mission

____

____

____

____

Anticipatory Strategic Planning

____

____

____

____

Stakeholder Buy-In

____

____

____

____

Leadership

____

____

____

____

Pillar Assessment:

____

____

____

____

Value Creation Pillar

       

Proper Portfolio

____

____

____

____

Product Performance

____

____

____

____

Cycle Time and Responsiveness

____

____

____

____

Value of Work in Progress

____

____

____

____

Pillar Assessment:

____

____

____

____

Quality Focus Pillar

       

Capacity for Breakthroughs

____

____

____

____

Continuous Improvement

____

____

____

____

Commitment to Quality

____

____

____

____

Structured Processes

____

____

____

____

Learning Environment

____

____

____

____

Quality of Research

____

____

____

____

Pillar Assessment:

____

____

____

____

Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 31
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 32
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 33
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 34
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 35
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 36
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 37
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 38
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 39
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 40
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 41
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 42
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 43
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 44
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 45
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 46
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 47
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 48
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 49
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 50
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 51
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 52
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 53
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 54
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 55
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 56
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 57
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 58
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 59
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 60
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 61
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 62
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 63
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 64
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 65
Suggested Citation:"CHARACTERISTICS AND METRICS FOR A WORLD-CLASS ARMY RESEARCH, DEVELOPMENT, AND ENGINEERING ORGANIZATION." National Research Council. 1996. World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization. Washington, DC: The National Academies Press. doi: 10.17226/5486.
×
Page 66
Next: CONCLUSIONS AND RECOMMENDATIONS »
World-Class Research and Development: Characteristics for an Army Research, Development, and Engineering Organization Get This Book
×
Buy Paperback | $47.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!