National Academies Press: OpenBook

Procedures and Guidelines for Validating Contractor Test Data (2020)

Chapter: Chapter 4 - Summary and Recommendations for Future Research

« Previous: Chapter 3 - Findings and Applications
Page 49
Suggested Citation:"Chapter 4 - Summary and Recommendations for Future Research." National Academies of Sciences, Engineering, and Medicine. 2020. Procedures and Guidelines for Validating Contractor Test Data. Washington, DC: The National Academies Press. doi: 10.17226/25823.
×
Page 49
Page 50
Suggested Citation:"Chapter 4 - Summary and Recommendations for Future Research." National Academies of Sciences, Engineering, and Medicine. 2020. Procedures and Guidelines for Validating Contractor Test Data. Washington, DC: The National Academies Press. doi: 10.17226/25823.
×
Page 50

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

49 C H A P T E R 4 Many SHAs use contractor test results in acceptance and payment decisions. One key step in this process is ensuring appropriateness of the contractor test data through validation. For example, the 23 CFR 637B permits the use of contractor test data for construction materials acceptance, as long as SHAs validate the contractor data with independent test results. There was a need to identify or develop statistically sound and practical procedures for validating contractor construction materials test data. These procedures need to address different appli- cations (materials and procurement types) and related issues, such as sample size, minor deviations, associated risks, and other constraints. This research was performed to recom- mend procedures for validating contractor test data for construction materials and to prepare guidelines for their application in the form of a proposed AASHTO practice. This research is described in Part I of the report; Part II presents the proposed practice for validating contrac- tor test data. The research included a review of literature, SHA standard specifications, ongoing research projects, and current practices relevant to procedures for validating contractor test data as well as a web-based survey of SHAs. The survey revealed that 22 of 28 responding SHAs use contractor test results as part of the acceptance procedure. The survey also identified challenges that SHAs face with QA programs and showed that the F- and t-tests are used much less than expected in comparison to other methods used by SHAs. The less fundamental, higher risk methods used by many SHAs supported the need for improved procedures and guidelines for validating contractor test data for construction materials to ensure conformance with the SHA standards and to manage associated risks. Building upon the findings from the literature review, assessment of the state of current practice and review of the fundamental statistics associated with available procedures, the research identified potential candidate validation procedures, recommended a set of procedures for validation of contractor test data, and developed a proposed AASHTO practice for SHAs for applying these procedures. Also, risk factors were identified, and mitigation measures to limit associated risks were proposed. Among the risk factors identified were bias, use of nonstatistical or statistically weak methods, data manipulation, sample size, and risks associated with violating the F- and t-tests assumptions. The research revealed that some SHA sampling and testing plans that use contractor data in acceptance decisions do not use independent samples and thus, do not meet the requirements of 23 CFR 637B, and some SHA sampling and testing plans use a single SHA sample per lot. The research team developed plans for sampling and testing of the SHA data and for contractor data validation. To address the SHA single sample per lot issue, a cumulative sampling lot technique was introduced. These plans are incorporated in proposed practice for validating contractor test data. Summary and Recommendations for Future Research

50 Procedures and Guidelines for Validating Contractor Test Data The numerical analysis provided the basis for recommending validation tests on a statistical basis. The observations were validated using SHA project data. The t-test and the Welch’s t-test showed consistent satisfactory results at the selected significance levels regardless of distribu- tion type. The F-test showed consistent satisfactory results at the selected significance level. The observations of both numerical and SHA data analyses supported using F-test and t-test. These tests consistently showed satisfactory results, but the Welch’s t-test showed more consis- tency in detecting the difference in means than the t-test and other hypothesis tests regardless of whether the variances were equal or not. The Welch’s t-test is an adaptation of Student’s t-test and is more reliable when the two samples have unequal variances and unequal sample sizes. Five examples were presented to illustrate use of the recommended procedures for different scenarios. The examples illustrated sampling method (split versus independent), outlier detec- tion, retesting or resampling and retesting, and validation versus nonvalidation of contractor test results. Project data obtained from SHAs were used in these examples. The research led to the following conclusions: • The QA program depends on whether the SHA will conduct the acceptance sampling and testing or use contractor data for acceptance sampling and testing. • QA programs need to meet certain minimum requirements, including validating contractor data with independent test results (similar to 23 CFR 637B SHA requirement). • Observations of both numerical and SHA data analyses support using F-test and t-tests for validating contractor test data. • The t-test and the Welch’s t-test consistently showed satisfactory results, and the Welch’s t-test showed more consistency in detecting the difference in means than the t-test and other hypothesis tests regardless of whether the variances were equal or not. To help construction and materials engineers effectively use contractor test results and to reduce the risk of faulty acceptance decisions and pay adjustments, a proposed practice was prepared to describe appropriate processes for validating contractor results and recommend subsequent actions when the results are validated or not validated. The proposed practice suggests incorporating the following items in the validation process of contractor test data: • F-test and unequal variance t-tests for primary validation. • Independent SHA and contractor samples. • Adequate sample sizes to manage SHA and contractor risk. • A cumulative sampling technique to address low SHA sampling rates. • Outlier detection prior to conducting F-tests and unequal variance t-tests. • Explicitly defining resampling and retesting provisions in specifications. • Paired t-test for secondary validation when primary validation fails. • Not using high-risk single test comparison methods (e.g., D2S and X− ± CR). Recommendations for Future Research The research proposed procedures for validating contractor test data. However, further research is needed to address several aspects of QA programs, such as sample size of the data sets and the associated risks to both parties for multiple AQCs; implementation tools (software) that may be used in QA programs to improve validation procedures; and procedures for validating contractor test data for alternative delivery projects.

Next: References »
Procedures and Guidelines for Validating Contractor Test Data Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In the development of a quality assurance program, a state highway agency (SHA) incorporates policies and procedures for assessing how well the materials and construction used by a contractor on a construction project satisfy the agency's specifications.

The TRB National Cooperative Highway Research Program's NCHRP Research Report 946: Procedures and Guidelines for Validating Contractor Test Data reviews and evaluates the procedures and practices used in the acceptance of construction materials. The report also proposes a related practice for incorporation into AASHTO Standard Specifications for Transportation Materials and Methods of Sampling and Testing.

Supplemental files to the report include Appendices A through E, available as a single PDF file, a State Highway Agency Survey (Appendix A), Survey Responses (Appendix B), Numerical Simulations (Appendix C), State Highway Agency Date Analysis (Appendix D), and Statistical Tables (Appendix E).

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!