Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
49 C H A P T E R 4 Many SHAs use contractor test results in acceptance and payment decisions. One key step in this process is ensuring appropriateness of the contractor test data through validation. For example, the 23 CFR 637B permits the use of contractor test data for construction materials acceptance, as long as SHAs validate the contractor data with independent test results. There was a need to identify or develop statistically sound and practical procedures for validating contractor construction materials test data. These procedures need to address different appli- cations (materials and procurement types) and related issues, such as sample size, minor deviations, associated risks, and other constraints. This research was performed to recom- mend procedures for validating contractor test data for construction materials and to prepare guidelines for their application in the form of a proposed AASHTO practice. This research is described in Part I of the report; Part II presents the proposed practice for validating contrac- tor test data. The research included a review of literature, SHA standard specifications, ongoing research projects, and current practices relevant to procedures for validating contractor test data as well as a web-based survey of SHAs. The survey revealed that 22 of 28 responding SHAs use contractor test results as part of the acceptance procedure. The survey also identified challenges that SHAs face with QA programs and showed that the F- and t-tests are used much less than expected in comparison to other methods used by SHAs. The less fundamental, higher risk methods used by many SHAs supported the need for improved procedures and guidelines for validating contractor test data for construction materials to ensure conformance with the SHA standards and to manage associated risks. Building upon the findings from the literature review, assessment of the state of current practice and review of the fundamental statistics associated with available procedures, the research identified potential candidate validation procedures, recommended a set of procedures for validation of contractor test data, and developed a proposed AASHTO practice for SHAs for applying these procedures. Also, risk factors were identified, and mitigation measures to limit associated risks were proposed. Among the risk factors identified were bias, use of nonstatistical or statistically weak methods, data manipulation, sample size, and risks associated with violating the F- and t-tests assumptions. The research revealed that some SHA sampling and testing plans that use contractor data in acceptance decisions do not use independent samples and thus, do not meet the requirements of 23 CFR 637B, and some SHA sampling and testing plans use a single SHA sample per lot. The research team developed plans for sampling and testing of the SHA data and for contractor data validation. To address the SHA single sample per lot issue, a cumulative sampling lot technique was introduced. These plans are incorporated in proposed practice for validating contractor test data. Summary and Recommendations for Future Research
50 Procedures and Guidelines for Validating Contractor Test Data The numerical analysis provided the basis for recommending validation tests on a statistical basis. The observations were validated using SHA project data. The t-test and the Welchâs t-test showed consistent satisfactory results at the selected significance levels regardless of distribu- tion type. The F-test showed consistent satisfactory results at the selected significance level. The observations of both numerical and SHA data analyses supported using F-test and t-test. These tests consistently showed satisfactory results, but the Welchâs t-test showed more consis- tency in detecting the difference in means than the t-test and other hypothesis tests regardless of whether the variances were equal or not. The Welchâs t-test is an adaptation of Studentâs t-test and is more reliable when the two samples have unequal variances and unequal sample sizes. Five examples were presented to illustrate use of the recommended procedures for different scenarios. The examples illustrated sampling method (split versus independent), outlier detec- tion, retesting or resampling and retesting, and validation versus nonvalidation of contractor test results. Project data obtained from SHAs were used in these examples. The research led to the following conclusions: â¢ The QA program depends on whether the SHA will conduct the acceptance sampling and testing or use contractor data for acceptance sampling and testing. â¢ QA programs need to meet certain minimum requirements, including validating contractor data with independent test results (similar to 23 CFR 637B SHA requirement). â¢ Observations of both numerical and SHA data analyses support using F-test and t-tests for validating contractor test data. â¢ The t-test and the Welchâs t-test consistently showed satisfactory results, and the Welchâs t-test showed more consistency in detecting the difference in means than the t-test and other hypothesis tests regardless of whether the variances were equal or not. To help construction and materials engineers effectively use contractor test results and to reduce the risk of faulty acceptance decisions and pay adjustments, a proposed practice was prepared to describe appropriate processes for validating contractor results and recommend subsequent actions when the results are validated or not validated. The proposed practice suggests incorporating the following items in the validation process of contractor test data: â¢ F-test and unequal variance t-tests for primary validation. â¢ Independent SHA and contractor samples. â¢ Adequate sample sizes to manage SHA and contractor risk. â¢ A cumulative sampling technique to address low SHA sampling rates. â¢ Outlier detection prior to conducting F-tests and unequal variance t-tests. â¢ Explicitly defining resampling and retesting provisions in specifications. â¢ Paired t-test for secondary validation when primary validation fails. â¢ Not using high-risk single test comparison methods (e.g., D2S and Xâ Â± CR). Recommendations for Future Research The research proposed procedures for validating contractor test data. However, further research is needed to address several aspects of QA programs, such as sample size of the data sets and the associated risks to both parties for multiple AQCs; implementation tools (software) that may be used in QA programs to improve validation procedures; and procedures for validating contractor test data for alternative delivery projects.