National Academies Press: OpenBook

State DOT Product Evaluation Processes (2024)

Chapter: Summary

« Previous: Front Matter
Page 1
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2024. State DOT Product Evaluation Processes. Washington, DC: The National Academies Press. doi: 10.17226/27809.
×
Page 1
Page 2
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2024. State DOT Product Evaluation Processes. Washington, DC: The National Academies Press. doi: 10.17226/27809.
×
Page 2
Page 3
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2024. State DOT Product Evaluation Processes. Washington, DC: The National Academies Press. doi: 10.17226/27809.
×
Page 3
Page 4
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2024. State DOT Product Evaluation Processes. Washington, DC: The National Academies Press. doi: 10.17226/27809.
×
Page 4
Page 5
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2024. State DOT Product Evaluation Processes. Washington, DC: The National Academies Press. doi: 10.17226/27809.
×
Page 5

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1   The construction, maintenance, and operation of transportation infrastructure require immense quantities of products and materials. New products, materials, engineered systems, and innovative technologies are presented to state departments of transportation (DOTs) by suppliers and manufacturers, as well as sometimes by contractors and internal DOT staff. However, any potential product must fill a need, be cost-effective, not negatively impact other construction and maintenance operations, and meet DOT specifications and func- tional requirements to be of value to the DOT and its operations. To determine the suit- ability of a product, state DOTs use Product Evaluation Programs (PEPs). A state DOT PEP includes processes, procedures, and tools for state DOTs to test and evaluate a product and ultimately decide whether it is approved for use. Approved products are added to the Approved/Qualified Products List (A/QPL). State DOTs revise specifications or create new specifications when one does not exist. Once on the A/QPL, the products are available to DOT staff and associated third-party contractors and consultants for use. State DOTs are government agencies that welcome, and are sometimes required by state law to accept, the submission of potential new products for evaluation. State DOTs depend on PEPs and associated processes and systems to evaluate products objectively, consistently, and timely. In addition, state DOTs have the ability to use many different products. How- ever, product evaluation processes differ among state DOTs based on agency operations. Therefore, information is needed to understand the current state of practice in product evaluation for transportation construction and maintenance. The objective of this synthesis was to document the current practices, funding, policies, management techniques, tools, and workflows of state DOT PEPs. The scope of the syn- thesis is confined to the state DOT PEPs and A/QPLs in use and the current state of prac- tice associated with the product evaluation of construction and maintenance products and materials. The synthesis followed a three-step process: (1) reviewing existing literature; (2) distrib- uting a national survey questionnaire to all 50 state DOTs and the Washington, DC, DOT; and (3) conducting case example interviews with the selected state DOTs. The literature review entailed the collection of academic and practical literature, documents, and reports and is detailed in Chapter 2. Using the literature reviewed, a national survey questionnaire was developed and distrib- uted to the members of the AASHTO National Transportation Product Evaluation Program (NTPEP) Technical Services Program and the AASHTO Committee on Materials and Pave- ments (COMP). A total of 42 state DOTs completed the survey questionnaire (an 82.3% response rate). Chapter 3 presents the survey results. S U M M A R Y State DOT Product Evaluation Processes

2 State DOT Product Evaluation Processes The case examples were conducted to emphasize the survey results and to dig deeper into state DOT PEPs and A/QPLs. State DOTs were selected for a case example interview based on whether they completed the survey, use a formal state DOT PEP, have a comprehen- sive A/QPL, and have a willingness to participate. Seven state DOTs (Arizona, California, Florida, Michigan, New Hampshire, Ohio, and Tennessee) participated in the case example interviews. The case examples focused on product evaluation processes and management; evaluation timelines; evaluating products that do not align with an A/QPL; PEP bene fits and challenges; A/QPL structure and use; evaluating proprietary, environmental, and Buy America/ Build America products; and lessons learned. Each case example is discussed in Chapter 4, and the interview questions used in each case example can be found in Appendix C. According to the literature review and survey results, all state DOTs have a PEP in place. Survey results also showed that about two-thirds of the responding state DOTs reported using a formal state DOT PEP and A/QPL, and one-third reported using a more informal approach. Regarding the structure and workflow of state DOT PEPs, the survey results indi- cated that the categories they used were based on either material types (e.g., asphalt, concrete, and sealants) or the suitability of being grouped with existing approved or qualified products. The case example interviewees mentioned the importance of product categories aligning with current specifications. Aligning product categories with specifications allows for easy refer- encing in product use, field inspection, testing requirements, performance measurement, and payment approaches. Regarding product submission and based on the state DOT’s guidelines, manufacturers, suppliers, and product representatives are allowed to submit an application for a product evaluation. According to the survey results, 93% (39) of the responding state DOTs stated that product manufacturers submit products for evaluation. Eighty-three percent (35) stated that product representatives submit products for evaluation. In some cases, third-party suppliers (33% or 14 of the responding DOTs), contractors (31% or 13), and consultants (14% or 6) may submit a product for evaluation, while internal state DOT personnel may only recommend a product for evaluation. Once a product application is received and the evaluation begins, state DOT PEP staff assign a subject-matter expert (SME) individual or group, internal DOT staff with expertise in specific products or product types, to the product application; however, these individuals have other responsibilities beyond reviewing products. The case examples and literature review showed that state DOTs have existing testing and performance criteria that must be met for product approval for submitted products with defined uses and that match existing specifications. Additional information may be requested from the manufacturer throughout the evaluation process. When a product is considered new or innovative, there is less structure to follow for the evaluation and selection of SMEs because those types of products do not align with an A/QPL, an existing specification, or the performance criteria. According to the case examples, once the evaluation is complete, a decision is made to either approve or not approve the product; approved products are then added to the A/QPL. Also, a letter is sent to the manufacturer acknowledging the product’s approval. For products not approved, a letter is sent to the applicant describing the reasons for rejection so that the manufacturer can attempt to improve the product’s performance and resubmit it for evalu- ation. According to the survey results and case examples, most state DOTs allow manufac- turers to resubmit their product as long as the manufacturer has addressed any product performance issues. The majority of products submitted for evaluation tend to align with the state DOT’s A/QPL or existing specifications and evaluation criteria. However, in some cases, state DOTs also receive new or innovative products that do not align with the A/QPL or an

Summary 3 existing specification. While the survey results and case examples did show that state DOTs evaluate products that do not align with the A/QPL, the process is not as defined and can take considerably more time than those products that do align with a state DOT’s A/QPL. The difficulty lies in determining the evaluation criteria to use for the product and potentially developing a new specification. Arizona DOT and the California Department of Transportation (Caltrans) both relayed that the development of specification and evalua- tion criteria is a lengthy process and requires more resources in terms of SMEs and funding for testing. Michigan DOT stated that the time needed to develop a new specification and evaluation criteria includes SMEs researching the product and its associated performance, which is not a part of the product evaluation process when a product aligns with the state DOT’s A/QPL. Florida DOT stated that it does not have one set process to evaluate new or innovative products due to variation in products and performance. State DOT PEP staff mentioned that guidance is needed to efficiently evaluate new or innovative products that do not align with state DOTs’ A/QPLs or existing specifications and evaluation criteria. State DOTs review various items during product evaluations, including technical work- sheets and submittal information, laboratory testing, field testing, pilot testing, and demonstra- tions for their product evaluations. In addition, electronic systems in the form of online applications, product databases, cloud-based systems, and others provide opportunities for better evaluation efficiency and timely completion of evaluations. Florida DOT has a pro- gram called Product and Application Tracking History (PATH) that is used for all product evaluations and automates portions of the evaluation. Another example is New Hampshire DOT’s NForm system, a database that tracks the submission of product submittal packets and uses automated notifications to inform staff of incoming product evaluation applica- tions. Arizona DOT has AZPEP, an electronic online submission portal for product appli- cations that also sends automated notifications to PEP staff and SMEs as needed. Electronic systems with automated workflows and notifications are beneficial but, from discussions in the case examples, require significant initial effort to create and implement. Approximately three-fourths of the state DOT survey respondents have policies to eval- uate products. The policies typically include processes and workflows to follow, as well as dictating what products and materials are evaluated. Minnesota DOT’s policy on product evaluation (see Appendix D) asserts that any material, product, or engineered system must meet all standards and requirements before being approved and added to the A/QPL. South Carolina DOT’s policy states that it will evaluate only those new products that have the potential to fulfill a real need, economically provide a satisfactory level of service, and are covered by existing specifications. It is the burden of the manufacturer or supplier to dem- onstrate that a need for a new product exists and that the product has good potential for satisfying that need. Sixty-nine percent (35) of the state DOT survey respondents indicated that the timelines of product evaluations vary depending on the product. The survey respondents also noted that typical evaluation timelines vary depending on the manufacturer or supplier and access to available information. Average timelines range between 30 days and 5 years. According to the state DOT PEP procedure manuals, a specific number of days is to be stated, such as receiving requested information from a manufacturer or supplier within 30 days, or the evaluation is terminated. The case examples showed that the products that align with a state DOT’s A/QPL or existing specifications are typically evaluated in shorter timelines than products that do not align with such specifications. Sixteen state DOT survey respondents reported having dedicated full-time staff to operate and manage the state DOT PEP and associated A/QPLs. Of these 16 respondents, 15 stated that the PEP staff consists of one to five individuals. Ohio DOT noted in its case example that

4 State DOT Product Evaluation Processes one person, the New Product Engineer (NPE), facilitates the entire state DOT PEP. The case example interviewees noted that dedicated staff aid in PEP efficiency, but subject-matter expertise is required from other business units of the DOT. In addition, only nine of the state DOT survey respondents had allocated annual funding to their state DOT PEP. The amount of annual funding reported was $80,000 or less. According to the survey results and case examples, state DOTs reported using general funds and project funds to support product evaluations. Also, some state DOTs require the applying manufacturer to obtain and pay for the laboratory and field testing of a product. The case example interviewees also noted that successful state DOT PEPs are dependent on the support of the agency and assistance from internal staff. The literature review and case examples indicated that using state DOT A/QPLs is an effective practice in building an efficient state DOT PEP. In fact, the literature review and survey responses noted that all state DOTs have an APL or QPL in place. A state DOT’s A/QPL provides a one-stop shop for contractors to consider certain products for projects. Contractors are able to easily find the products they are allowed to use. In addition, internal state DOT staff know how to use the A/QPL to collect information on products submitted by contractors. While the structure of A/QPLs varies among state DOTs, it is typically based on product type categories that align with specifications and associated evaluation criteria for ease of use so that users know when and how the approved product can be used. The number of categories also varies; according to the case examples, A/QPLs may include a couple dozen categories (e.g., Ohio DOT) or a few hundred (e.g., Florida DOT). NTPEP resources include evaluation and audit programs and associated technical com- mittees, such as DataMine, which is an online repository of data and audit reports for all NTPEP services, and the Unique, Patented, Proprietary Products (UP3) program, which assists DOTs in the evaluation of innovative and proprietary transportation products. The survey results showed that most state DOTs use the AASHTO NTPEP, DataMine, and UP3 for innovative products. DOTs cited the following as the main reasons for using NTPEP: 1. Increases efficiency in product evaluation, 2. Prevents duplication of evaluation effort, 3. Produces lower evaluation costs through shared information, and 4. Reduces the time needed to evaluate products for use. However, NTPEP does not establish performance requirements, pass/fail or approve/ disapprove products, or make any determination about the suitability or acceptability of products. According to the case examples, the New Hampshire and Tennessee DOTs rely heavily on NTPEP and associated resources for conducting product evaluations. The other case example state DOTs mentioned limited use of NTPEP or using it for specific products. Arizona DOT stated that 10 product types currently rely on NTPEP resources for conduct- ing the evaluation. According to the case examples, state DOTs also noted using A/QPLs from other state transportation agencies, but only in rare circumstances and done with care. Ohio DOT men- tioned its use of another state agency’s A/QPL, but has only done so once in the past 5 years, and the product is still subjected to Ohio DOT’s evaluation criteria, which may be different from other state DOTs. Florida DOT noted that its climate is drastically different from other states, and that limits the information it is willing to use from other state DOTs’ A/QPLs. The New Hampshire and Tennessee DOTs tend to look at neighboring state DOTs’ A/QPLs on rare occasions, but this is done to review information, not to approve products. The differences in product types, applications, and performance thresholds among state DOTs make it difficult to share and use other state DOTs’ A/QPLs.

Summary 5 Once a product is approved and added to the A/QPL, state DOTs have varying timelines regarding the maintenance of that approval status. According to the survey results, 60% (25) of the responding state DOTs indicated that approved products remain on their A/QPL until removed, and removal typically is due to the product becoming obsolete, showing defects, no longer filling a need, or no longer providing the value it initially did. The other 40% (17) of the responding state DOTs indicated that a re-evaluation or recertification process must occur 1 to 6 years after the initial approval. The case example interviewees also men- tioned that re-evaluation occurs typically every 1 to 5 years, if a manufacturer changed a product, or if the state DOT modified an existing specification. State DOTs noted that product re-evaluations typically follow the same full process as a new product evaluation. Knowledge gaps found in this study represent areas for further exploration. Findings show that guidance is needed to develop consistent practices in product evaluations. The various methods in place for product evaluation at state DOTs do not allow for much sharing of information or approval of products that other states have already approved. Also, while NTPEP does provide vital information, it does not set the testing limits or tolerances for acceptance. Assistance with how to set appropriate tolerances for products, especially those that do not align with an existing A/QPL or specification, is needed. Finally, some state DOTs noted their use of an electronic system to manage the PEP processes. Guidance could be devel- oped on how to create and manage an electronic PEP system that streamlines and makes the process more efficient.

Next: Chapter 1 - Synthesis Overview »
State DOT Product Evaluation Processes Get This Book
×
 State DOT Product Evaluation Processes
Buy Paperback | $91.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The construction, maintenance, and operation of transportation infrastructure requires immense amounts of products and materials. New products, materials, engineered systems, and innovative technologies are presented to state departments of transportation (DOTs) by suppliers and manufacturers, as well as sometimes by contractors and internal DOT staff.

NCHRP Synthesis 616: State DOT Product Evaluation Processes, from TRB's National Cooperative Highway Research Program, documents current state DOT practices, funding, policies, management techniques, tools, and workflows of product evaluation processes.

Supplemental to the report is a dataset of various administrative documents from different state DOTs.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!