Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
8 Performance Management of Recruiters Earlier chapters described various approaches that can be used in an evaluation framework for decision making. Econometric, experi- mental, qualitative, and survey approaches all have their place in this framework. We have seen how each of these approaches has strengths and weaknesses that play a role in determining which approach (or ap- proaches) to use in a particular situation. Many of our examples have focused on recruiting resources and how best to use them. For example, we have addressed evaluation of advertising themes, the appropriate levels of joint and Service-specific advertising, various incentives (and the levels of incentives needed) to attract recruits, and others. In this chapter, we focus on what Barnes, Dempsey, Knapp, Lerro, and Schroyer (1991) refer to as "the linchpin to recruiting success" the recruiter. Recruiter performance management encompasses the range of issues and decisions that face Service recruiting managers as they organize to meet their mis- sion. We demonstrate that effective performance management requires multiple evaluation methods. ISSUES IN PERFORMANCE MANAGEMENT Service recruiting managers establish systems to select recruiters from among the available pool of Service members, to train and develop those new recruiters, to open recruiting offices in specific locations, to establish production goals for each recruiter, to motivate recruiters with reward and recognition programs, and to monitor and assess recruiter performance. Many models and options are available for each of these 146
PERFORMANCE MANAGEMENT OF RECRUITERS 147 systems, and each model or option chosen is open to evaluation. In some cases (for example, selection of new recruiters), there are continuing research programs to evaluate the effect of alternative programs. In other cases (for example, the effects of recruiter reward and recognition pro- grams), research or evaluation is rarely attempted. In still other cases (for example, establishing recruiter goals), there is a research base for some aspects of the program (for example, determining market size) but very limited research on the effectiveness of alternative approaches to actually setting goals. Given the central role that recruiter productivity plays in the recruiting process, all aspects of recruiter performance management should be subjected to evaluation efforts. Performance management systems are certainly affected by the envi- ronment in which the recruiter works. (By environment, we mean all of the conditions that surround recruiters performing their jobs including the environmental conditions created by the military and those that are broader and culturally based.) As stated in the committee's earlier report, the achievement of recruiting goals can be highly dependent on the eco- nomic conditions of the time, with high unemployment rates resulting in the easier attainment of recruiting goals. Moreover, other environmental factors, such as well-publicized military actions, proximity to military bases, and recruit's acquaintance with soldiers, may have some impact on a recruiter's performance. Despite the importance of these environmental factors, we have not addressed them specifically. In many cases, we rec- ommend that researchers look for general principles that can be applied regardless of the environmental conditions. In others, we encourage mili- tary researchers to take environmental factors into consideration when appropriate. The issues we address in this chapter involve all aspects of the evalu- ation framework introduced in Chapter 1. Our framework will guide us to appropriate evaluation methods for a given situation or specific aspect of performance management. Because recruiter performance management systems including both existing and proposed new programs are con- cerned with recruiter attitude, intentions, and behavior, multiple evaluation methods will have a role. Thus, in evaluating performance management systems, there will be situations in which experiments, or econometric techniques, or qualitative techniques are most appropriate. APPROACHES TO EVALUATION OF RECRUITER PERFORMANCE MANAGEMENT In many ways, military recruiting is similar to other kinds of sales activities in the civilian sector. While the military recruiting environment includes many features that distinguish it from the civilian sales environ-
148 EVALUATING MILITARY ADVERTISING AND RECRUITING meet, it is not so unusual that knowledge gained in the civilian environ- ment should be ignored. There is a substantial body of research on perfor- mance management of sales forces. When those research findings are coupled with an examination of best practices used in high-performing sales organizations, military recruiting management is presented with numerous alternatives to current practices. In choosing among those alter- natives, recruiting management can design and carry out studies using the techniques described in earlier chapters. In the remainder of this chapter, we describe some approaches that could be considered in evaluating the effectiveness of recruiter performance management alternatives. We review areas in which such approaches may be productive, not to describe in detail how such evaluations should take place. Many of the topics discussed selection, training and develop- ment, reward and recognition programs, and performance assessment- have extensive research literatures that deal with many of the problems the Services face, and the Services should seek out appropriate expertise in designing and executing evaluations of such programs. Recruiter Selection The problem of selecting people who will become successful sales- people is not unique to the military, yet there are elements of this selection problem that are found only in the Services. One aspect of staffing the military recruiting function that presents a substantial challenge is the pipeline of potential recruiters from which the military chooses. The Services have specific needs for recruiting personnel and have generally concluded that uniformed Service personnel will be detailed into recruit- ing positions. The pool of potential recruiters thus consists of both volun- teers and nonvolunteers, who have already been trained in some other military specialty and have some track record of success in that other specialty. In some respects, then, the military may know more about its potential recruiters than most organizations know about their candidates for sales positions. However, organizations hiring salespeople can look for people with previous (successful) sales experience and can assume that the vast majority of applicants actually desire to be hired. Neither the strategy of looking for previous experience in sales nor the assumption of motivated applicants is available to the military. Although the Services have traditionally used enlisted personnel as recruiters, Congress recently mandated experimental use of civil- ian recruiters in the National Defense Authorization Act for Fiscal Year 2001. These experiments are not yet complete; however, the results of such studies should be taken into account when selection issues are considered.
PERFORMANCE MANAGEMENT OF RECRUITERS 149 We have previously noted that the Services today do not always give major weight in the recruiter selection process to a candidate's potential for success in a sales environment (National Research Council, 2003~. Similarly, the U.S. General Accounting Office noted that the Services typi- cally focus on past job performance in nonrecruiting (i.e., nonsales) posi- tions when selecting recruiters (U.S. General Accounting Office, 1998~. Thus, more efficient and effective methods for choosing specific personnel who should be assigned as recruiters are likely to exist. There is a substantial literature, both military and civilian, addressing the problem of selecting people for sales occupations generally and mili- tary recruiting positions specifically. For example, Vinchur, Schippmann, Switzer, and Roth (1998) reviewed 97 studies of the relationship between predictors and job performance of salespeople. They conclude that per- sonality dimensions, tests designed specifically for predicting sales suc- cess, individual interests, and other biographical information items are useful in selecting people for sales occupations. Recent reviews of the literature discuss many possible approaches that could increase the likeli- hood of successfully selecting recruiters who will be high producers (Penney, Horgen, and Borman, 2000; Penney, Sutton, and Borman, 2000b; McCloy et al., 2001~. Similarly, there is substantial guidance in the profes- sional literature on appropriate ways to evaluate the effectiveness of selection systems (American Psychological Association, 1987~. The process for determining the effectiveness of a selection procedure is relatively straightforward and well understood. Although there are several approaches to establishing the effectiveness selection procedures (i.e., the extent to which a selection procedure predicts all or part of required job performance), a common approach involves statistically relat- ing test scores and job performance measures. Researchers begin by con- ducting a job analysis often employing multiple methods, such as focus groups, interviews, and surveys and using its results to identify the tasks that are performed by incumbents and the knowledge, skills, abili- ties, and other characteristics (KSAOs) necessary to perform those tasks. They then search for existing measures (such as tests and interviews), adapt existing measures, or develop new measures of those KSAOs. Pre- vious research indicates that cognitive ability, various personality traits, and vocational interests may be appropriate constructs to be measured in recruiter selection tools. Often, a combination of such measures proves to be the best predictor of job performance (Borman, Toquam, and Rosse, 1978~. Candidate selection measures are administered to some number of applicants for a position (or incumbents on a job), performance data are collected after some period of time on the job, and the relationship between performance on the measures and performance on the job is
150 EVALUATING MILITARY ADVERTISING AND RECRUITING determined. Experimental (or quasi-experimental) designs are used to ensure that inferences from the study can be interpreted. Managers should be aware that they need not invest immediately in multiple, large-scale studies that disrupt their routine processes in order to sort through the many options that are available for selecting recruiters. Small-scale studies can be very helpful in eliminating options or identify- ing those options with high promise. In addition, some of the Services have pursued this line of research extensively. McCloy et al. (2001) provide a recent example that implements this general approach. They focused on estimating the value of a cognitive abilities test (in this case, the Armed Services Vocational Aptitude Battery or ASVAB) and recruiter school grades as predictors of the quantity and quality of recruiter productivity. Their analyses controlled for additional factors that can affect recruiting productivity, such as the number of young men and women in the local population and the number of high schools in the vicinity. They then added an evaluation of the cost-effectiveness of a particular selection approach; that is, they considered the trade-off between the cost of implementing a selection system and the gains expe- rienced from increased productivity if the system is implemented. McCloy et al. concluded that adding ASVAB scores and other available demo- graphic information is a cost-effective method for improving the selection of Navy recruiters. Two issues mentioned above must be addressed when carrying out this type of selection research. First, appropriate measures of recruiter performance are required. Choices available here illustrate the difficulty of determining which performance measure or measures to use. For example, number of contracts in some period of time might be appropri- ate and is often used as a starting point in the process. However, markets vary in size and propensity; thus, number of contracts is often adjusted to consider those factors. Similarly, not all contracts are alike. Some recruits are more desirable than others; thus, number of contracts is often quali- fied by specifying "high-quality" contracts. Finally, number of contracts clearly does not capture all aspects of a recruiter's job. Once a contract is signed, administrative requirements must be attended to, and recruiters must continue to motivate applicants until they actually arrive at their basic training location. For example, Borman, Hough, and Dunnette (1976) identified eight dimensions of military recruiter perfor- mance: (1) locating and contacting qualified prospects; (2) gaining and maintaining rapport; (3) obtaining information from prospects; (4) sales- manship skills; (5) establishing and maintaining good relationships in the community; (6) providing knowledgeable and accurate information; (7) administrative skills; and (8) supporting other recruiters and the Recruit- ing Command. From this list, perhaps only dimensions 1 through 4 are
PERFORMANCE MANAGEMENT OF RECRUITERS 151 being considered directly in a performance measure that is based on number of contracts. If dimensions 5 through 8 are also important dimen- sions of recruiter performance, then the number of contracts must be supplemented with other performance measures. In sum, appropriate performance measures must be carefully chosen to represent those per- formance dimensions being targeted by the proposed selection measure. The second issue is relatively rare in the civilian selection arena, in which most applicants desire the job to which they are applying. Specifi- cally, those people being considered for the job of military recruiter do not always actually want the job. Thus, the evaluation strategy described above will be useful for identifying measures that will help select volun- teers who will be most productive, but those same measures may not be as useful in selecting among nonvolunteers. People with the KSAOs to perform the job may not have the motiva- tion to do so in an environment in which the job is particularly difficult and the rewards are low. Thus, the development and validation of selec- tion procedures that identify individuals with the KSAOs to perform the job (particularly those procedures that focus on cognitive skills) may be inadequate if they do not identify those with the willingness to perform at the high levels needed by the Services in a perceived low-reward environ- ment. Thus, in addition to considering the total context of recruiter per- formance, selection procedures must also better address the motivational aspects of job performance. It is important to remember that the selection problem should not be considered out of context. Military recruiting is not just another sales job. Moreover, the environment in which recruiters work can have a profound effect on motivation. To some extent, motiva- tion to perform the job might be better addressed by manipulating the reward and recognition structure as well as the work environment in which the recruiter performs rather than by developing better selection tools. Recruiter Training and Development Once recruiters are selected, they must be trained to perform their jobs, and they must continue to be developed as they progress in their recruiting career. As with most training requirements, there are multiple strategies available to address the recruiter training requirement, and there are constrained training budgets that limit the amount of time (and thus the specific tasks) for training. As structured today, the Services offer a combination of full-time, in-residence training along with on-thejob training to try to ensure that recruiters have the skills required to be successful on the job. In evaluating the overall effectiveness of training, recruiting management must address two central issues. The first concern
152 EVALUATING MILITARY ADVERTISING AND RECRUITING is the extent to which the training provided adequately covers the entire recruiting job. The second concerns the effectiveness of the training given. While not an area for research, a third issue related to effectiveness of training that should be mentioned is accessibility of training ensuring that the right training is available at the right times in a recruiter's assignment. Determination of training requirements typically begins with a needs analysis that specifies what skills are required to adequately perform a job and includes an analysis of where and how those skills might be acquired. Processes for both steps are well documented in the profes- sional training literature. Unless the training is directed at developing the appropriate skills for the job, it is unlikely to be effective. As with recruiter selection, there is a substantial professional litera- ture addressing the issues involved in the development of recruiter train- ing programs, both in the civilian environment and in the military. The committee has recommended that the Services develop and implement training systems that make maximum use of realistic practice and feed- back (National Research Council, 2003~. The Federal Advisory Committee on Gender-Integrated Training and Related Issues (1997) also included recommendations for improving recruiter training. The second issue is evaluating training effectiveness. Is a specific training approach accomplishing its goals? How effective is it compared with alternative approaches? There is also an extensive professional lit- erature regarding training evaluation in both the civilian environment and the military. While Salas, Milham, and Bowers (2003) note that rigor- ous training evaluations are rarely completed for military training courses, that does not mean that such evaluations should not be completed or cannot be completed. On the contrary, training evaluation is usually cited as a critically important aspect of the instructional systems design process used by the Services. The need to evaluate recruiter training programs has not been ignored. Hull and Benedict (1988) proposed an evaluation methodology for Army recruiter training; Hull, Kleinman, Allen, and Benedict (1988) carried out an evaluation based on that methodology. Those authors used outcome variables that included ratings from instructors and both current and former students. Chonko, Madden, Tanner, and Davis (1991) used a qualitative approach to evaluate the effectiveness of the specific sales techniques taught to Army recruiters. Kirkpatrick (1976) described four levels of training evaluation: (1) reaction, (2) learning, (3) behavior change, and (4) results. When Salas et al. note the lack of rigorous training evaluations, they are focusing on evaluations at levels 3 and 4 of the Kirkpatrick model. Frequently, survey methods are used to determine whether trainees like
PERFORMANCE MANAGEMENT OF RECRUITERS 153 (or dislike) a particular training course, and end-of-course tests (coupled with tests given at the beginning of a course) determine whether trainees increased their knowledge in a subject area. While trainee reactions and mastery of specific knowledge are both important aspects of military recruiter training (or sales training in general), more critical outcomes of this training include whether or not the trainee can apply what he or she has learned (behavior change) and whether or not the trainee becomes an effective recruiter (results). Appropriate evaluation of training programs often requires the use of multiple strategies and techniques. As noted above, survey techniques may be used as a component of training evaluation to determine trainee and instructor reactions to the training. Surveys can determine whether trainees liked the training, believed that the training improved their skills, believed that the training would be applicable on their jobs, believed that time devoted to particular tasks was appropriate, and so forth. Similarly, surveys of trainees and their supervisors provide information useful in guiding training course design and revision. Surveys can be used to determine whether supervisors believe that the training syllabus includes the proper tasks or skills that trainees need, that time devoted to particu- lar tasks is appropriate, and so forth. At the same time, experimental and quasi-experimental designs play roles in evaluating whether training results in a specific behavior change or a change in the overall effectiveness of job performance. While it may be inconvenient or expensive to implement true experimental designs (with treatment and control groups and random assignment to condi- tions), preexperimental or quasi-experimental designs can provide valu- able information in the overall evaluation of training programs (Sackett and Mullen, 1993~. Outcome measures for these studies must be carefully chosen. We noted the Borman et al. (1976) dimensions of recruiter perfor- mance in our discussion of recruiter selection issues. The choice of appro- priate criteria against which to evaluate recruiter training is a similarly complicated issue. If the eight Borman et al. dimensions are all determi- nants of successful recruiter performance, then it follows that they are all candidates for inclusion in the evaluation of recruiter training programs. Choosing only a single outcome measure, such as number of new con- tracts in some fixed period of time, as the standard for evaluating recruiter training that covers all aspects of the recruiter job provides little informa- tion that would be useful in improving the training course. It is important to remember that development takes place in ways other than formal training programs. Often, individual feedback and coaching around certain experiences is a very effective way to shape behavior. Experiential learning and associated coaching assume that there are capable coaches who understand what the desired behavior is and
154 EVALUATING MILITARY ADVERTISING AND RECRUITING who can communicate performance deficits and strategies for improve- ment. Thus, several potentially productive activities may include the Services studying the informal development of recruiters and identifying ways to ensure that those who supervise recruiters possess adequate knowledge of the recruiting process as well as coaching skills themselves. Recruiting Office Locations Once the Services have selected and trained recruiters, the question arises, "Where should recruiters be located?" As with training, the answer to this question is always constrained by budget, by policies that dictate combining offices from multiple Services when feasible, etc. Along with the location of each specific office, there are also questions addressing the number of recruiters who should be stationed in that office. Each of these decisions is open to evaluation, with the goal of establishing the most effective recruiting organization for a given budget, or with the goal of minimizing the cost of obtaining a given number of recruits. There is little systematic research on strategies for locating recruiters. However, existing data lend themselves to econometric analyses. For example, recruiter productivity over time should be available for existing offices. That productivity can be modeled by such available data as size of qualified recruiting market in the office's geographic region, experience levels of the recruiters assigned to that office, incentive programs (for recruits or for recruiters) that were available at given times, and so forth. Such analyses could guide recommendations for appropriate placement of recruiting offices. Through econometric modeling, the Services should minimally be able to define the relevant variables for establishing recruit- ing locations and to consider various methods for determining optimal staffing levels. Recruiter Production Goals Most sales organizations establish goals (targets or quotas) for perfor- mance of their sales force. Many variables potentially shape these goals. For example, some organizations take into account the "product" being sold as well as the area in which it is sold and past demand for the product. Some may even take into account the experience and expertise of the salesperson. Military recruiting is no exception to this model. The question of how best to establish recruiter goals (and the question of whether those goals should be individual or team based) is still open. Given that recruiting duty is often cited as an extremely stressful job- due to the constant pressure to "make goal" there would seem to be
PERFORMANCE MANAGEMENT OF RECRUITERS 155 high payoff in defining the variables relevant to military recruiting goals and evaluating the goal-setting process. Recruiter Reward and Recognition Programs Sales organizations generally have well-established reward and rec- ognition programs, closely tied to performance as measured against goals. Here as well, military recruiting organizations are little different from their civilian sales organization counterparts (although military recruiting organizations must contend with legal constraints against using financial incentives for recruiters). The hallmark of many sales reward and recog- nition programs is pay and bonuses based on sales performance. Pay for recruiters is determined by Congress and cannot be adjusted for this job. Because the military cannot use most types of financial incentives, the Services rely instead on nonfinancial incentives such as plaques, watches, rings, and military decorations in lieu of cash compensation. However, the effectiveness of such incentives as a reward for past performance and a subsequent motivator of future performance remains an open question. The Federal Advisory Committee on Gender-Integrated Training and Related Issues (1997) made a number of recommendations for changes in recruiter incentive systems. For example, they suggest that the overall level of recruiter incentives needs to be increased, that the Services should make a recruiting assignment career-enhancing, and that a recruiter's rewards and recognition should be linked to his or her recruits' perfor- mance in basic training. All of the evaluation approaches raised in this report should be con- sidered when attempting an evaluation of these incentives. For example, there will often be adequate existing data to support econometric analyses of the effects of specific programs on recruiter behavior as measured by productivity over some period of time. Asch (1990) provides such an example. She used demographic data and varying incentive structures to estimate effects on effort and productivity, concluding that rewards and the timing of rewards affect the allocation of recruiter effort over time. In the absence of existing data, experiments could be devised to determine the effect of alternative programs on recruiter intentions (as measured by surveys) or recruiter behavior (as measured by productivity). In addition to evaluating the effectiveness of existing incentives, the Services must also consider other approaches to reward and recognition. Defining the most appropriate array of alternatives must be based on review of the existing compensation literature and analysis of the needs and expectation of the recruiting work force. The Services could use focus groups and interview techniques to generate a list of alternative incen- tives that might be expected to motivate recruiter effort and understand
156 EVALUATING MILITARY ADVERTISING AND RECRUITING the trade-offs among them. For example, the Services may find that recruiters are willing to accept low cash incentives if sales (i.e., recruiting performance) are rewarded by career enhancement. The Services could also explore the acceptability and potential incen- tive value of various career enhancements. We note the widespread per- ception that successful duty as a recruiter or drill instructor is necessary for advancement to senior noncommissioned officer rank in the Marine Corps. In addition, there have been suggestions that recruiter special duty assignment pay which is now based exclusively on length of time as a recruiter could be structured to provide an incentive for production rather than an incentive for staying in the recruiting job. Interestingly, there are some financial disincentives associated with the recruiter's job. Anecdotally, recruiters often recount the financial hard- ships of living in areas without a substantial military presence. The lack of services often provided on military bases and the cost of medical care are often cited. Thus another approach to rewarding recruiters is removing the financial disincentives. Recruiter Performance Assessment Answers to any of the questions raised above from recruiter selec- tion through recruiter reward and recognition programs assume that there are some measures of recruiter (or team or organizational) perfor- mance against which alternatives can be compared. These performance assessment measures themselves are also open to evaluation in a number of ways. First, the Services must define the critical aspects of the recruiter job. The Borman et al. (1976) study defined eight dimensions of recruiter per- formance; however, we must note that the study is almost 30 years old. Since many things about the Services and the recruiting environment have changed in the past 30 years, we could reasonably assume that the recruiting task has also changed. In addition, enlisted military recruiters are also part of their respective Services. Those dimensions of performance associated with simply serving must also be included if the performance assessment is intended to cover the entire job. A second task is to assess the accuracy of the appraisal itself. Numer- ous studies have noted that military performance measures tend toward leniency that is, disproportionately large numbers of individuals receive high ratings. Similar problems exist in civilian performance appraisal sys- tems. The use of experimental performance measures in test development and validation work also suggests that typical measures are not adequate. Consequently, the Services must continue to develop and employ perfor- mance appraisal systems that provide accurate ratings of job performance.
PERFORMANCE MANAGEMENT OF RECRUITERS 157 The military has already devoted considerable resources to the devel- opment of performance appraisal systems. Theoretically, at least, the prob- lem may not be with the performance appraisal system but with the environment in which it is used. Thus, a third avenue of exploration is studying the conditions under which performance assessments are made and used. Because the recruiting job is so unlike other military jobs, the Services might take the approach of developing a performance appraisal system that is outside the routine systems used for administrative pur- poses. Such a system might have feedback to the recruiter as its primary purpose, rather than rating or ranking recruiters. Another area of research the Services may find productive is to link performance assessment to training and development options. If the Services are able to identify specific areas of weaknesses and provide a feedback mechanism, then follow-up suggestions for remediation will be required to actually improve the skills of the recruiting workforce. Experi- mental research will be necessary to identify development opportunities that actually rectify performance deficits. A final area of evaluation is the effectiveness of communications about the performance appraisal system. Stating what is to be rewarded is one step in increasing the probability that such behavior will be exhibited. If performance expectations are greater than simply the number of con- tracts, then that information needs to be clearly communicated to recruiters so that they can manage their performance in light of a broader view of performance. Similarly, comprehensible, individual feedback on perfor- mance is another critical step in ensuring that recruiters understand their own strengths and weaknesses and are able to target their own develop- mental efforts appropriately. CONCLUSIONS In many respects, the problems of performance management faced by the military are no different from the problems faced by private industry. However, the environments are distinctly different, and the military faces many restrictions to which the private sector is not subject. Because of these environmental differences, some of the existing research from the professional literature will be useful, some not. Ideally, the military should undertake continuous and systematic evaluation of each aspect of perfor- mance management individually and as a whole in order to improve recruiter performance, relying on professional literature when possible, undertaking its own research when necessary. Research that considers the interactions among various factors that affect recruiting performance is particularly needed. However, to under- take such massive research may be both overwhelming and practically
158 EVALUATING MILITARY ADVERTISING AND RECRUITING and economically infeasible. Nevertheless, doing nothing heightens the likelihood of recruiting problems ranging from difficulty assessing the more qualified candidates to expenditures disproportionate to the results achieved. The challenge for the Services will be to establish their own overall research frameworks, prioritize their many options, select those research options that are most promising, and continually revise the research plan based on findings and changes in the environment.
