5
DETAILED STEPS IN THE DEVELOPMENT AND IMPLEMENTATION OF PERFORMANCE STANDARDS1
The workshop’s second day began with a presentation by Guy Mulder, Executive Director of Veterinary and Professional Services at CRL and Attending Veterinarian for North American Research Models and Services. Mulder described how CRL, a leading commercial breeder of laboratory animals, develops performance standards. CRL, he said, has developed a standardized process for creating performance standards, including a proposal process and a dedicated form that proceeds step by step, creates consistency, and provides value in the future when reevaluations are conducted. He noted the company started performance approach evaluations in 1997, and when he joined the company a decade ago he was able to go through the company files to review the history of each performance evaluation, which included the data sets the company used to make its decisions at each evaluation along the way.
At CRL, the guiding principle is that if the company deviates from methods outlined in the Guide, the resulting conditions for the animals should be equivalent or better. If, however, the changes create conditions that do not meet minimal Guide recommendations, then any deleterious changes in animal welfare should be measurable and documented. The challenge, Mulder explained, is to develop the appropriate measurable parameters to enable a valid evaluation of the alternative method.
When the 8th edition of the Guide was published, CRL conducted a gap analysis to identify exceptions to the Guide and determine where it did and did not comply with Guide recommendations. “There are many things that many of us do at our institutions that are historical - we’ve
_________________________
1 This section is based on the presentation by Guy Mulder, and the statements are not endorsed or verified by the National Academies of Sciences, Engineering, and Medicine.
done them for 10, 20, or even 30 years - and the Guide suggests we do things otherwise. I think we need to take a moment and evaluate why we are doing it differently and if it is time to change our practices,” said Mulder. If the answer to an evaluation shows there is a scientific, operational, or welfare-oriented reason not to adopt the Guide’s recommendation, he said, there is then an opportunity to develop and validate a performance standard to address that recommendation in the Guide.
Developing a performance standard requires first defining the desired outcomes and goals and conducting a review of the current literature and industry best practices to identify worthwhile performance measures to evaluate the alternative method. The next step is to design, plan, and perform an evaluation, which at CRL includes talking to experts and auditors to gain insights into methods they use at their facilities. This evaluation, which has been reviewed and approved by the IACUC, includes directly comparing the alternative approach to the Guide recommendations, something many institutions do not do based on what Mulder has learned through his conversations with experts and auditors. The IACUC then reviews the results of the evaluation and either approves, disapproves, or requires modifications to the alternative approach. Once approved, a post-approval monitoring plan is enacted with an annual review.
The key stakeholders in this process, explained Mulder, include the IACUC, which is involved from the very beginning; the attending veterinarian; the researchers who will be using the animals; any specialists with unique expertise in areas of study design, equipment, or interpretation of the study results; and the vivarium management, including the husbandry personnel, who often have an intimate knowledge and unique insights that he might not have as the veterinarian.
Mulder said that CRL has created a dedicated Request for Exception to ILAR 2011 Guide Recommendations form, which he characterized as a research proposal, complete with hypothesis and proposed statistical analysis. This form requires a description of the specific procedure or practice forming the exception, the species affected, the Guide recommendation on the subject, the proposed exception, the rationale for the exception, and the proposed performance measures and methods to investigate the exception. For the cage density guideline exception, this variance request was 18 pages long, and it summarized the relevant literature and included CRL’s efforts in the area.
Once the IACUC approves the proposal and the studies are completed, the results are summarized on the form and it goes back to
the IACUC for review and approval. When the variation is approved, there is a final section of the form for the post-approval monitoring program the IACUC must also approve. Examples of variances or exceptions to the Guide for which CRL has used a performance approach include cage density for mice and rats, cage change frequency, housing multiple rodent species in the same room, and the frequency of water bottle sanitation. For each of these, Mulder’s team developed evaluation criteria unique to the question they were asking.
When the 8th edition of the Guide was published, one of the concerns for the company with regard to housing density was the new category of “female plus litter,” which did not exist in the 7th edition. Taking a literal application of this recommendation, which was that there be one female plus litter plus a breeding male in one cage, CRL was concerned that the standard cage it uses might not always allow the space allocation described in the Guide, Mulder explained. In addition, for many of CRL’s mouse strains, the company’s practice was to have a second female in the same cage so that the male will breed with two females, creating a communal group in the cage. Such an arrangement would not be appropriate in CRL’s cages according to the Guide’s density requirements. “To transition away from that breeding scheme would have meant adding a significant number of new cages to our program and building new space simply to achieve the same production, the same output that we were achieving prior to the release of the new Guide,” said Mulder. The new recommended minimum space requirements challenged the company to ask if there were data showing its existing breeding densities were detrimental to the animals, he added.
The application for variance Mulder’s team developed applied only to production facilities housing mice and rats, and it specifically excluded experimentally manipulated animals. Mulder noted that the company has modified cage densities over the years in response to internal evaluations, professional judgment, and changes in the knowledge base, and he expects there will be modifications in the future as well. The rationale for the variance included explicit documentation for why the company believes this variance is necessary. The rationale also included internal data showing that animal welfare is not compromised at current cage densities used by CRL and that existing space allocations meet the definition of adequate space. The variance proposal also noted that the Guide states there is a lack of peer-reviewed literature establishing specific space allocations for rodents in breeding setting.
Given the new recommendations in the Guide, CRL proposed repeating some of the studies it had previously conducted but with more
measures or indices than it had used in the past. For example, to better assess whether current densities meet a requirement to provide sufficient space for mothers and litters to allow pups to develop to weaning without detrimental effects for mother or litter, the proposed studies needed to assess pup development and maturation as well as the health of the mother. Mulder and five of his colleagues debated which performance measures needed evaluating and ultimately decided that the performance measures should include a range of categories of animal health, behavior, and production indicators and the data should be collected in the standard production setting so as to not add confounders into the evaluation or complicate post-approval monitoring.
The group considered a number of practical issues and decided that the performance measures should not rely on extensive instrumentation, such as telemetry readings for heart rate as a measure of stress, or complex behavioral assessments. It also decided to restrict the evaluations to a representative number of stocks and strains, rather than performing the evaluation for every stock and strain in the company’s breeding program, and to a limited number of the company’s facilities providing the environments are similar, as they are throughout the company’s many North American facilities. The result was a set of measures assessable in an active breeding area and some that needed to be done in a behavioral testing setting.
Mulder and his colleagues then considered what likely measureable events could arise from selecting the alternative housing density and selected the measures they expected would most likely be affected if there were adverse effects. With respect to cage space for a mother with litter or a breeding group, the group adopted reproductive indices, growth, general health, aggression, and lack of stereotypic behaviors as reasonable outcomes. If the performance standard was about cage change frequency, the appropriate measures might involve humidity in the cage, ammonia levels, and other gas production from waste products, Mulder explained.
For reproductive performance, Mulder’s team started with a list including the number of pups weaned per female breeder per week, known as the production index; litter size at birth; survival to weaning; sex ratio; weaning weight; inter-litter interval; time to first plug; and time to vaginal opening in female pups. Mulder noted that there is published literature addressing each of these for the stocks and strains CRL commonly uses. For behavioral assessment, the biggest concerns were about space utilization within the cage and aggression or fighting, either maternal aggression towards the pups or aggression between weanlings
and adults. Other concerns included hair loss, stereotypic behavior, and space utilization with the cage. The CRL team developed a scoring system for each parameter, most of which could be measured in the barrier room by standard husbandry technicians who received some additional training from the company’s behavioral staff. Assessments of space utilization, however, were performed in a behavioral lab, which Mulder characterized as a quiet room outfitted with video equipment that could simultaneously record 24 hours a day from 48 mouse cages and 32 rat cages, spanning both light and dark cycles and multiple litters of breeding groups. Clinical measures included morbidity and mortality, animals euthanized for cause, body conditioning scoring, and growth curve comparisons from weaning through age 10 weeks.
As an example of the scoring system, Mulder described the behavioral assessment scoring for fighting and aggression (Figure 5-1), barbering (Figure 5-2), and stereotypic behaviors (Figure 5-3). Staff was trained to recognize these behaviors so they could be consistent in how they scored them while performing their normal daily or weekly routines with the animals. Data were collected, collated, and sent to the behavioral staff for evaluation.
FIGURE 5-1 Charles River Laboratory’s behavioral assessment scoring for fighting and aggression
SOURCE: Mulder slide 22
FIGURE 5-2 Charles River Laboratory’s behavioral assessment scoring for barbering
SOURCE: Mulder slide 22
FIGURE 5-3 Charles River Laboratory’s behavioral assessment scoring for stereotypic behaviors
SOURCE: Mulder slide 23 (adapted from Würbel and Stauffacher 1996)
After careful consideration of which stocks and strains, or genotypes, to evaluate the CRL team picked the C57B/6, the most common inbred mouse strain used in research and common background strain for genetically engineered mice; CD-1, or Swiss mouse, the most common outbred mouse; the Sprague-Dawley or CD rat, an outbred strain; and the brown Norway rat, an inbred strain. These strains, said Mulder, reflected
the majority of mice and rats CRL produces as well as the majority of those being utilized in research.
The proposal submitted to the IACUC was put together by a larger team that included veterinarians, a behavioral scientist with significant rodent experience, IACUC members, and production management and husbandry staff. Every team member was going to be involved in the study and so had an opportunity to help plan the study. After the proposal received IACUC approval and the team started collecting data, team members discussed what to do if the data showed there were differences in outcome measures between the groups. A difference, Mulder explained, could identify a problem, or it might reflect the animals are adapting to a new cage without harm to their welfare. For example, housing room temperature for rodents is generally below the thermal neutral zone, but the rodents can adapt if they can huddle together or burrow into bedding or nesting materials.
The experimental design for the cage density evaluation consisted of a Latin square factorial design, with cage type or size, breeding condition, and strain of animal being the three variables. For mice, the breeding conditions consisted of breeding the female and then removing the male so the female is alone with the pups; a matched male and female pair kept together, including when the litter is present; and a trio of two females and one male. For rats, there were two breeding schemes: pair a male and female together for a brief period of time and then remove the male; or a male-female pairing in which the pair is kept together.
The decision about cage size proved to be more complicated than expected. Initially, the plan was to compare the current CRL cage with one meeting the 2011 Guide specifications. However, after reviewing the literature and discussing that idea with the behavioral specialist, it seemed doubtful there would be any significant differences between the two groups because the cages were not far apart in size. Instead, the team selected the smallest and the largest cages available on the market to compare with the current CRL cage.
The cage sizes used with the mice were (Figure 5-4):
- 226 cm2, the smallest cage available
- 305 cm2, the standard CRL cage
- 432 cm2, a Guide-compliant cage
- 800 cm2, the largest cage available
FIGURE 5-4 The smallest and largest mouse cages
SOURCE: Mulder slide 32
For rats, the cage sizes used were (Figure 5-5):
- 580 cm2, the smallest cage available
- 758 cm2, the standard CRL cage
- 903 cm2, a Guide-compliant cage
- 1355 cm2, the largest cage available
FIGURE 5-5 The smallest and largest rat cages
SOURCE: Mulder slide 33
With the final results to be published soon, Mulder presented some preliminary findings, one of which was that cages can be too small. While reproductive performance in mice did not decrease in the smallest cages, which Mulder termed a surprise, CD rats showed a drop in reproductive performance in the smallest cage. There was also excessive soiling in the smallest cage. Mulder said the smallest mouse cage with a female and litter to be weaned was obviously overcrowded and everyone agreed it would never be used in an ongoing breeding program.
There was only one significant difference on the performance indices between the CRL standard cage, the Guide-compliant cage, and the largest cage. The one behavioral change observed in adults was something that had not been reported in the literature, and the investigators called it “corner inactivity.” This behavior was only seen at specific pup ages, typically two weeks of age and older, and it was seen in all cages, though it was more frequent in smaller cages. It may be, said Mulder, that females may benefit from vertical space that allows them to escape the older pups.
The full results were submitted to the IACUC, which after some discussion approved the continued use of the internal CRL cage densities for breeding of mice and rats. Mulder noted that the study itself took six months to complete, followed by another six months of watching video to obtain scores on the behavior measures. Though the study was perhaps more thorough than it needed to be, he said the data were reassuring. For post-approval monitoring the IACUC will conduct an annual review of key performance indicators for the variance for two consecutive years and then do a de novo review at the end of year three with a complete literature review to determine if new or additional studies are warranted. There is also regular ongoing review of production indices that is outside of direct IACUC involvement. This review will involve both management and Mulder or one of his colleagues and will look for differences or drift in the indices that might be an early sign of a developing issue that would then require IACUC involvement.
Diane Gaertner asked if the cages were open, static or ventilated, and Mulder replied this study used open cages, which are the standard cages used in production setting barrier rooms. She then asked if the study included monitoring room ammonia or other room parameters, and Mulder replied that those measures were not part of this study, though
room ammonia is monitored on a quarterly basis in all of CRL’s barrier rooms.
Norman Peterson from MedImmune asked Mulder for his opinion as to when the published body of knowledge becomes large enough to change the standard of practice. Mulder said he does not entirely agree that there need to be site-specific evaluations for every facility and every institution, and that if there are publications showing generally consistent findings in several different settings, those findings should be generalizable. He added, however, that for cage density, the literature is not all that clear and so for this issue more studies in different settings are still needed before a new practice standard is established. He did note that “perhaps there should be a more common approach to evaluating questions such as housing density so that the results of those studies are more comparable, but at some point we need to be able to say ‘this seems reasonable,’ and groups such as AAALAC should be ready to agree that individual studies don’t need to be performed at every single institution.”
Paul Locke asked how another facility might go about generalizing from the publications resulting from the studies Mulder described. The first thing to do, Mulder replied, is to look at the performance measures in the publication and see if they fit the proposed study. One item he noted specifically that might make it difficult to generalize from the CRL study was the use of open top cages in a large production setting, something that other facilities are unlikely to use. He envisioned future studies looking at ventilated housing that could be more generalizable across other ventilated caging systems.
David Kurtz said that he hoped one of the goals of this workshop and for future activity is to encourage those in the community to publish their data and disseminate it, making it possible to judge when a particular performance standard works in multiple different settings. He also suggested that until a new edition of the Guide is published or revisions are made to the AWR, data from these performance standard studies could be used to adjust the engineering standards to be more appropriate and realistic. He then asked Mulder if the post-approval monitoring process has identified any strains for which their results on their model strains and species may not apply. Mulder replied that for routine, non-disease models, there have not been any issues identified so far. Disease models, such as for obesity and diabetes, will require a different approach to housing density, cage cleaning, and other parameters, but those studies have not been conducted.
David Anderson posed a question from an online participant, who asked if the presence of engineering standards has inhibited the development of performance standard. In Mulder’s opinion, they have because the easiest route is to take the engineering approach rather than developing a justification and rationale for developing a performance standard.
Another question from an online participant asked Mulder about the time and resources required to develop and complete this project. He replied that CRL made a significant investment in both time and resources. The company hired a Ph.D. behaviorist, primarily to assist with designing, performing, and evaluating the study, and it purchased specialized equipment. The study itself took a year for data collection and analysis and another six months to prepare the publications to go out to the peer-reviewed literature. In fact, said Mulder, not many institutions have the internal resources to do this type of extensive study. His hope is that as the literature base grows, subsequent studies could be scaled back to produce smaller data sets that will align to the published studies and can justify site-specific practices.
