National Academies Press: OpenBook
« Previous: 1 Setting the Context
Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×

Science-Based Assessment and Experimental Design

SCIENCE-BASED TESTING FOR COMBINATION DEVICES

Aric Kaiser

Center for Devices and Radiological Health

U.S. Food and Drug Administration

Before a new medical device can be marketed, the Center for Devices and Radiological Health at the U.S. Food and Drug Administration (FDA) must review its risks and effectiveness. A device is defined as an apparatus or implant intended for the diagnosis, mitigation, treatment, or prevention of disease that does not achieve its primary intended purposes through chemical action and that is not dependent on being metabolized. The review process for traditional devices, familiar to most device manufacturers, requires that a manufacturer demonstrate one of two sets of device classification criteria:

  • There must be reasonable assurance of the safety and effectiveness of the device based on preclinical and clinical evaluations; or

  • The device must have the same intended use as a predicate device, along with either the same technological characteristics or different technological characteristics that do not raise different questions of safety and effectiveness.

Combination devices combine with drugs and/or biologic components to effect a treatment. The review process for these products may be different from that for traditional devices, but generally, although some additional regulatory requirements must be fulfilled, the questions that a manufacturer must address as part of a preclinical or clinical evaluation of a combination product are essentially the same. For example, in the evaluation of any device that is implanted or that has contact with tissues, the materials used in the device must be assessed to determine whether they can safely be used or implanted in the human body (biocompatibility). When a biologic component is combined with a device, an additional determination must be made regarding whether the materials used are free from potentially infectious agents.

Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×

SCIENCE-BASED TESTING FOR BIOLOGICS

Darin J. Weber

Center for Biologics Evaluation and Research

U.S. Food and Drug Administration

In general, biological products are complex mixtures of multiple components. Because they are prepared from living sources, special consideration must be given to preventing the transmission or introduction of infectious agents while preserving product identity, purity, and potency. In order to ensure that no infectious agents are present, U.S. Food and Drug Administration (FDA) regulations for biological products include specific science-based testing of the final product.

FDA’s General Biological Product Standards (21 CFR 610) include standards for manufacturing safety (sterility, mycoplasma, purity, adventitious viral agents) and for the assessment of product characteristics such as identity, viability, and potency. In addition to testing the final product, safety testing and other assessments are performed throughout manufacturing in order to evaluate the manufacturing process itself and to ensure that the quality and consistency of the product lots are maintained.

Standard test methods are of limited use, however, when testing biological products consisting of living cells. Many of the prescribed tests, such as sterility tests, take days or weeks to complete, and may thus limit the development of products that cannot be stored for such long periods of time. FDA has therefore adopted a flexible approach that allows some products to be used clinically even if the final test results are not available. FDA’s Center for Biologics Evaluation and Research also supports and encourages the development of alternative test methods (as described in FDA standard 21 CFR 610.9) to meet the need for fast, sensitive, and reliable test methods for these products.

USING DESIGN OF EXPERIMENT METHODS IN THE INNOVATION PROCESS

James Rutledge

DataVision Statistical Consulting and Training, LLC

It is often said that “knowledge is power,” and gaining knowledge is what design of experiments (DOE) methods are all about. Effectively researching, developing, and maintaining a product requires all the knowledge that can

Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×

reasonably be obtained about the process used to make that product. A process is defined here as a series of inputs and outputs. Inputs include the various aspects of the process that might influence the resulting product, such as reaction time and temperature, material thickness, vendor source, and lot variations. Knowing the impact on the end product of variance in the inputs is important. Outputs are measurable quantities that describe product characteristics or performance, such as the size of an extruded part or the yield of a chemical process. A mathematical understanding of how process inputs relate to outputs results in profound process knowledge and the power to control and improve processes.

It is very important to have quantifiable measures for inputs and outputs, and the systems used for making those measurements must be repeatable and reproducible. For example, in a measurement system, an engineer would perform a gauge repeatability and reproducibility study. In a chemical system, a scientist would validate the analytical method used for detection to ensure that it is sensitive, repeatable, and reproducible. There are industry standards for many of these tests, although new quantitative or semi-quantitative methods might have to be developed for histological evaluations of inflammation, for example.

The typical alternatives to DOE methods are “best-guess” and “one-factor-at-a-time” (OFAT) methods, both of which are based on the researcher’s strong understanding of the system to be evaluated. However, these intuitive approaches may not result in the correct answer or may not be understood and believed by others. They may miss important events, such as the interaction of one input variable with another and how that affects the product output. DOE methods, by contrast, allow the importance of the various inputs to be summarized quantitatively and allow the development of a mathematical model to run simulation experiments of the expected product when the process is run under various conditions.

Modeling is important because it allows us to understand the factors that influence the robustness of the process. This understanding is critical to initial process validation as well as ongoing manufacturing. It is also important in the up-front development process, where it can be used to refine decisions about how to operate the process before experiments are undertaken. This mathematical process, or in silico experimentation, is helpful in identifying process specifications that can eventually be verified in confirmation studies.

The modeling process will be most valuable if it is undertaken both to make the product to the desired specifications (targeting the process center) and to manage product variability within acceptable limits (reducing process variation). Controlling product variability is crucial for keeping the process under control and ensuring that every unit produced will fall within acceptable limits of the desired target.

Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×

One attribute of DOE experiments that is often overlooked is their efficiency in determining the importance of the inputs and developing the process models with a minimum number of experiments. In fact, DOE techniques require fewer resources than traditional methods because of the efficiency of the design. This is counterintuitive because designing and executing DOE studies seems to take longer. While the up-front planning stages of DOE are often more involved than those of traditional techniques, the efficient design used to analyze information results in the need for fewer experiments, thus saving time and resources. In addition, because more time is invested in planning the experiments, they are more likely to be definitive in their results. DOE experiments can also be used to identify those factors that have no impact on product target or variability. Decisions can then be made about eliminating such factors, or reducing control of those variables in order to save process costs.

In summary, DOE methods improve understanding of what influences the product process, both in hitting the target and in controlling variability; help inform decisions about how to run the process, through the modeling approach; and provide in-depth knowledge about interactions and the quantitative influence of inputs on product performance and characteristics. While computer software can simplify the analysis component of DOE studies, the experimental discipline and creative thinking of the engineer or scientist are what make DOE methods successful.

MAKING IT FLY: CURRENT BOEING CERTIFICATION PROCESSES

Stephen G. LaRiviere

Boeing Commercial Airplanes

When Boeing Commercial Airplanes develops a large transport aircraft, numerous certification processes must be successfully completed. The Federal Aviation Administration (FAA) of the Department of Transportation is the agency with oversight responsibility for these certification processes; Boeing Commercial Airplanes interacts primarily with the FAA Aircraft Certification Service, Transport Airplane Directorate, Manufacturing Inspection Office, Seattle Aircraft Certification Office, and Seattle Manufacturing Inspection District Office. A designated engineering representative (DER) is a Boeing employee who represents the FAA with the agency’s concurrence and who facilitates the certification process. The DER is generally a senior engineer who has both technical and communication skills and who is well respected by both the FAA and Boeing colleagues.

Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×

Federal aviation requirements are incorporated into all Boeing policies and procedures. Boeing must obtain type certificates (design approval for each airplane to be manufactured), production certificates (approval to build airplanes and airplane parts in accordance with the type design), and airworthiness certificates (approval to deliver and operate an airplane that has been built and tested in compliance with the type and production certification). Airplane type certification covers four areas: structures, materials, systems, and propulsion. For Boeing structural certification, a building block approach is used, with testing on the coupon, element, detail, subcomponent, and component levels. For materials certification, efforts are currently being made to achieve faster qualification through the use of critical-chain project management and, to a limited extent, design of experiments.

Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×
Page 9
Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×
Page 10
Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×
Page 11
Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×
Page 12
Suggested Citation:"2 Science-Based Assessment and Experimental Design." National Research Council. 2004. Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices. Washington, DC: The National Academies Press. doi: 10.17226/11035.
×
Page 13
Next: 3 Bone Morphogenetic Proteins and Orthopedic Repair »
Proceedings from the Workshop on Science-Based Assessment: Accelerating Product Development of Combination Medical Devices Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Food and Drug Administration (FDA) has established testing centers for assessment of three categories of medical products; devices, biologics, and drugs. Increasingly, however, medical products are appearing that are combinations of more than one of those categories. The FDA is just beginning to develop procedures for evaluating such combination products, which pose new challenges for assessing efficacy and safety. The Roundtable on Biomedical Engineering Materials and Applications (BEMA) is an NRC activity that brings together relevant parties to discuss R&D, applications, and regulation of biomedical materials and devices. In 2003, BEMA held a workshop to discuss science-based assessment for evaluation of combination products. This report and accompanying CD present abstracts and viewgraphs of the talks given at that workshop. The workshop focused on three specific types of combination products: orthopedic repair using morphogenetic protein, drug-eluting stents, and cell-matrix cartilage implants. In addition, context-setting discussions of science-based assessment and experimental design were presented at the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!