Simple Performance Tester for Superpave Mix Design: First-Article Development and Evaluation (2003)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5CHAPTER 2 FINDINGS In this section of the report, the key findings of NCHRP Project 9-29 are summarized. Detailed documentation of the each of the various project activities is presented in the appro- priate appendix to this report. Discussion of the practical ramifications of these findings is given in Chapter 3 of this report, while extension of the findings to general conclusions and recommendations are presented in Chapter 4. 2.1 KEY ELEMENTS OF THE FIRST-ARTICLE SPECIFICATION The basic philosophy behind the first-article specifications was to produce performance type specifications that would allow manufacturers to propose innovative design concepts to address issues associated with user-friendliness, cost, and reli- ability. The specifications described how the test must be con- ducted, what needed to be measured, and the accuracy and resolution of the measurements. Manufacturers were then per- mitted to propose various alternatives that met the specifica- tion requirements. Four first-article equipment specifications were developed during Phase I of the project. The follow- ing specifications were included as appendices in the Interim Report: • First-Article Equipment Specification for Specimen Fab- rication Equipment, • First-Article Equipment Specification for the Flow Time Test, • First-Article Equipment Specification for the Flow Num- ber Test, and • First-Article Equipment Specification for the Dynamic Modulus Test. Draft specifications were first prepared by the Project 9-29 research team based on the findings from the review of the Project 9-19 and 1-37A Draft Test Protocols and the work- shop conducted with materials testing experts in Task 1. These draft specifications were then sent to 13 potential man- ufacturers for review and comment and were reviewed in detail with 10 of the potential manufacturers at the manufac- turer’s workshop conducted in Task 3. The draft equipment specifications were revised based on comments obtained from the manufacturers and presented in the Interim Report. The Interim Report also included the recommendation that Phase II be directed at the procurement and evaluation of equip- ment capable of performing the flow time, flow number, and dynamic modulus test over a temperature range of 20 to 60 °C. This recommendation was approved by the project panel, and a final first-article equipment specification was prepared by combining elements from the test-specific specifications listed above. The final specification, “First-Article Equipment Spec- ification for the Simple Performance Test System,” is included in this report as Appendix A. The sections that follow sum- marize key elements of this specification. 2.1.1 Test Capabilities The simple performance test system can perform the three candidate uniaxial and triaxial compression tests recom- mended in NCHRP Project 9-19 (1): flow time, flow number, and dynamic modulus. These three tests are variations on tests that have been used for many years by various researchers to characterize asphalt concrete materials (2). The tests are per- formed on nominal 100-mm (4-in) diameter, 150-mm (6-in.) high cylindrical specimens cut and cored from over-height 150-mm (6-in.) diameter gyratory specimens. The simple per- formance test system includes a confining pressure system and environmental control over the temperature range of 20 to 60 °C. The tests are briefly described below. 2.1.1.1 Flow Time Test The flow time test is a variation on the simple compressive creep test that has been used by several researchers to mea- sure the rutting potential of asphalt concrete mixtures (3). In this test, a static load is applied to the specimen, and the resulting strains are recorded as a function of time. The vari- ation introduced by the Project 9-19 research is the concept of flow time, which is defined as the time when the minimum rate of change in strain occurs during the creep test. It is deter- mined by differentiation of the strain versus time curve. Fig- ure 1 presents an example of a typical creep response and the computation of the flow time. In this case, the flow time is

6approximately 155 sec, and the axial strain at the flow time is approximately 1.0 percent. In Project 9-19, the flow time correlated well with the rut- ting resistance of mixtures used in experimental sections at MNRoad, WesTrack, and the FHWA Pavement Testing Facil- ity (2). For tests at a given temperature, axial stress, and con- fining stress, the rutting resistance of the mixture increases as the flow time increases. Guidance on temperatures, stress levels to be used in the testing, and the minimum flow time needed to achieve acceptable rutting performance are the subject of ongoing research in Project 9-19. 2.1.1.2 Flow Number The flow number test is a variation on the repeated load permanent deformation test that has been used by several researchers to measure the rutting potential of asphalt con- crete mixtures (3). Figure 2 shows a schematic of the repeated loading used in this test. Haversine axial compressive load pulses are applied to the specimen. The duration of the load pulse is 0.1 sec followed by a rest period of 0.9 sec. The permanent axial deformation measured at the end of the rest period is monitored during repeated loading and converted to strain by dividing by the original gauge length. The vari- ation introduced by the Project 9-19 research is the concept of flow number, which is defined as the number of load pulses when the minimum rate of change in permanent strain occurs during the repeated load test. It is determined by differentiation of the permanent strain versus the num- ber of load cycles curve. Figure 3 presents an example of a typical permanent axial strain response and the computa- tion of the flow number. In this case, the flow number is 1300 and the permanent axial strain at the flow number is approximately 1.0 percent. In Project 9-19, the flow number correlated well with the rutting resistance of mixtures used in experimental sections at MNRoad, WesTrack, and the FHWA Pavement Testing Facil- ity (2). For tests at a given temperature, axial stress, and con- fining stress, the rutting resistance of the mixture increases as the flow number increases. Guidance on temperatures, stress levels to be used in the testing, and the minimum flow Figure 1. Typical creep test response and flow time. Figure 2. Schematic of flow number test loading.

number needed to achieve acceptable rutting performance are the subject of ongoing research in Project 9-19. 2.1.1.3 Dynamic Modulus Test. The dynamic modulus simple performance test recom- mended by the NCHRP Project 9-19 team is a variation on ASTM D3497. In this test, a continuous haversine axial com- pressive load is applied to a specimen at a given temperature 7 and loading rate. Measured stresses and strains are used to calculate the resulting dynamic modulus and phase angle. Fig- ure 4 presents a schematic of typical data from the dynamic modulus test. The dynamic modulus and phase angle are defined by Equations 1 and 2, respectively. (1) (2) where |E*| = dynamic modulus σ0 = amplitude of applied sinusoidal loading ε0 = amplitude of resulting sinusoidal strain φ = phase angle in degrees Tl = time lag, sec Tp = period of sinusoidal loading, sec Two dynamic modulus test procedures were recommended in Project 9-19: one for permanent deformation and one for fatigue cracking. The primary difference in the tests is the temperature for measuring the dynamic modulus. For per- manent deformation, tests will be performed at high temper- atures while an intermediate temperature will be used for the fatigue tests. In Project 9-19, dynamic modulus test results at 37.8 and 54.4 °C correlated well with the rutting resistance of mixtures used in experimental sections at MNRoad, Wes- Track, and the FHWA Pavement Testing Facility (1). The rutting resistance of the mixtures increased as the dynamic modulus at high temperatures increased. Guidance on test tem- peratures and the minimum moduli needed to achieve accept- able rutting performance are the subject of ongoing research φ =    × T T i p 360 E * = σ ε 0 0 Figure 3. Typical repeated load test response and flow number. Figure 4. Typical dynamic modulus test data.

in Project 9-19. The Project 9-19 research found only a fair correlation between cracking observed in the experimental sections and the dynamic modulus at 4.4 and 21.1 °C (1). Guidance on test temperatures and moduli needed to achieve acceptable fatigue performance are also the subject of ongoing research in Project 9-19. 2.1.2 Overall Test System Requirements The first-article specifications place specific requirements on the size, power requirements, and noise level for the simple performance test system. These were included in the specifi- cations to promote future implementation of the equipment in HMA plant laboratories. Table 1 summarizes key operational requirements included in the specification. 2.1.3 Compression Loading Machine The first-article specifications require a compression load- ing machine with closed loop control that can apply constant, ramp, sinusoidal, and pulse loads. Table 2 summarizes the required capacities of the loading machine. The specifica- tions do not specify the type of loading machine, but place specific requirements on the machine’s ability to control the loading. Table 3 summarizes the load control require- ments. For sinusoidal and pulse loads, a control require- ment was placed on the standard error of the applied load, defined by Equation 3. The standard error is a measure of how well the loading device reproduces sinusoidal loading, which is critical to the correct measurement of the dynamic modulus. (3) where se(P) = Standard error of the applied load xi = Measured load at point i xˆi = Predicted load at point i from the best fit sinusoid xˆo = Amplitude of the best fit sinusoid n = Total number of data points collected during test. Two types of loading machines were proposed by manu- facturers who responded to the Project 9-29 request for pro- se P x x n x i i i n o ( ) ( ˆ ) % ˆ = − −  = ∑ 2 1 1 100 8 posals: servo-hydraulic, and a unique linear motor based on electromagnetic control technology. As discussed in a later section of this report, both of the machines selected as the first-article equipment were servo-hydraulic. The first-article specifications required two configurations for the loading platens. For the flow time and flow number tests, the loading platens are required to remain parallel. For the dynamic modulus test, the loading platens must include a ball or swivel joint on one platen to allow that platen to con- form to the contour of the specimen end. Finally, loads must be measured with an electronic load cell. To minimize the potential for damage, the full-scale range of the load cell was specified to be at least equal to the stall force of the actuator. The accuracy of the load cell was specified to be ±1 percent over a load range of 2 to 100 per- cent of the capacity of the machine. The resolution of the load cell was specified to be that required by ASTM E4 (4), which is 1/100 of the lowest calibrated load. For the simple per- formance test devices with 10 kN capacity, the first-article specifications require a resolution of 2 N (0.5 lb). 2.1.4 Deformation Measuring Systems Two separate deformation measuring systems are required by the first-article specifications. The first, used in the flow time and flow number tests, measures the movement of the loading actuator. Measuring the flow time and flow number using an actuator-mounted system is a departure from the specimen-mounted instrumentation specified in Project 9-19 (1). Given that the flow time and flow number are determined from the derivative of the strain history, the Project 9-29 research team suggested that an actuator-mounted deforma- tion measuring device could be used to detect flow in these tests. Such instrumentation would simplify the equipment and technician skill level required to perform the flow tests. As long as machine compliance errors are not a function of time, they do not affect the computation of the flow time or flow number. This hypothesis was verified by additional test- TABLE 1 Key operational requirements for the simple performance test system TABLE 2 Compression loading machine capacities

ing conducted in Project 9-19, which compared flow times and flow numbers measured with three systems: • The specimen-mounted LVDTs specified in the Project 9-19 test protocol, • The radial LVDTs specified as an alternate in the Proj- ect 9-19 test protocols, and • An actuator-mounted LVDT. 9 Figure 5 shows data presented in the Project 9-19 June 2001 Quarterly Progress Report (5). This data confirmed that flow times and flow numbers from an actuator-mounted deformation system were the same as those from the specimen-mounted deformation system. The primary concerns in conducting the two flow tests are (1) whether or not the range of the actuator- mounted deformation measuring system is sufficient to obtain flow and (2) whether or not the resolution is adequate to allow detection of flow through numerical differentiation of the strain versus time curve. A minimum range of 12 mm (0.5 in.) and resolution of 0.0025 mm (0.00001 in.) was included in the first-article specifications. The second deformation measuring system is a specimen- mounted system used in the dynamic modulus test. The sys- tem specified in the first-article specifications differs from that described in the Project 9-19 test protocols in two ways. First, the gauge length was reduced from 100 mm (4 in.) to 70 mm TABLE 3 Load control requirements Figure 5. Comparison of flow from different instrumentation systems (5).

(2.75 in.). The shorter gauge length was specified for the sim- ple performance test system in an attempt to reduce variabil- ity caused by the instrumentation being mounted close to the ends of specimens with ends that are not perfectly smooth and parallel. In Phase I, measurements on a large number of specimens showed that some of the specimen dimensional tolerances included in the Project 9-19 test protocols could not be achieved with standard laboratory saws and drills. The tolerances in the Project 9-19 test protocols were based on those presented in AASHTO T231(6) for capping concrete cylinders. The revised tolerances listed in Table 4 were rec- ommended and incorporated in the first-article specifica- tions for an automated sawing and coring device. The use of a shorter gauge length in the dynamic modulus test is sup- ported by the findings of a large specimen size and geome- try study conducted as part of Project 9-19 (7). This study concluded that 70-mm (2.75-in.) diameter specimens with 70-mm (2.75-in.) gauge length could be used to measure the dynamic modulus of mixtures with nominal maximum aggre- gate sizes up to 37.5 mm. The second difference between the first-article specifica- tions and the Project 9-19 test protocols is the first-article specifications do not require the specific mounting system shown in the Project 9-19 test protocols (1), although the Proj- ect 9-19 mounting system can meet the requirements of the first-article specifications. This change was made to encour- age the consideration by equipment manufacturers of alter- native mounting systems that may simplify the equipment 10 and technician skill level required to perform the dynamic modulus test. To further encourage simplification of the instru- mentation and the possible use of noncontact sensors, the first-article specifications require rapid installation of the specimen-mounted measuring system so that specimen instru- mentation, installation in the testing machine, application of confining pressure, and temperature equilibration can be com- pleted in 3 minutes. Other requirements of the specimen- mounted deformation measuring system are listed in Table 5. 2.1.5 Confining Pressure The simple performance test system includes a confining pressure system for performing the tests with confinement. Table 6 summarizes the requirements of the confining pres- sure system. As discussed in the previous section, a time limit of 3 minutes for specimen instrumentation, installation, appli- cation of confining pressure, and temperature equilibrium was included in the first-article specifications to encourage manufacturers to use automated pressure cells. 2.1.6 Environmental Chamber The simple performance test system includes modest envi- ronmental control over a temperature range from 20 to 60 °C. This temperature range is based on the Project 9-19 recom- TABLE 4 Project 9-29 specimen dimension tolerances TABLE 5 Specimen-mounted deformation measuring system requirements

mendation that the simple performance tests for permanent deformation and fatigue cracking be conducted at the effec- tive temperatures for these distresses as defined by Equa- tions 4 and 5 (10). Teff (PD) = 30.8 − 0.12Zcr + 0.92(MAAT + KασMAAT) (4) where Teff(PD) = effective temperature in °C for permanent deformation Zcr = critical depth in mm for the mix layer in ques- tion MAAT = mean annual air temperature in °C Kα = value computed from normal probability table related to designers selected level of reliability. σMAAT = standard deviation of the mean annual air tem- perature. Teff (FC) = 0.8(MAPT) − 2.7 (5) where Teff(FC) = effective temperature in °C for fatigue cracking MAPT = mean annual pavement temperature in °C at one third of the depth of the pavement layer For the United States, the effective temperature for perma- nent deformation ranges from 25 to 55 °C and the effective temperature for fatigue cracking ranges from 12 to 20 °C. Table 7 summarizes the requirements of the environmen- tal chamber. As discussed in previous sections, a time limit of 3 minutes for specimen instrumentation, installation, application of confining pressure, and temperature equilib- 11 rium was included in the first-article specifications to encour- age manufacturers to use environmental chambers with suf- ficient capacity to reach equilibrium quickly. 2.1.7 Computer Control and Data Acquisition Computer control and electronic data acquisition were specified for the simple performance test system. The first- article specifications require the control software to include logic that prompts the user through each of the tests. The software includes on-line help and allows the user to choose either SI or US Customary units. For each of the simple per- formance tests, the first-article specifications include require- ments on the following: • Test control: the sequence of operations, parameters to be controlled during the test and their tolerances, and actions to be taken if control parameter tolerances are exceeded; • Data acquisition: data to be acquired and sampling rates; • Operator input: test identification information, test con- trol information, and remarks; and • Data storage and output: format for data files and hard copy reports. The first-article specifications also require real-time graph- ical display of information that will be useful to the operator. Displays for the flow time test include time histories of stress, strain, and the rate of strain. The flow number test includes a digital oscilloscope for real-time display of stress and strain as a function of time and histories of the peak stress, perma- nent strain, and permanent strain rate as a function of the number of load cycles. Finally, the dynamic modulus test TABLE 6 Confining pressure system requirements TABLE 7 Environmental chamber requirements

includes a digital oscilloscope for real-time display of the stress and strain measured during the test. 2.1.8 Computations The first-article specifications provide detailed algorithms for computation of the flow time, flow number, and dynamic modulus. The algorithms are much more specific than the general descriptions for data analysis included in the Project 9-19 test protocols (1). Important computational issues are summarized below for the three tests. 2.1.8.1 Flow Time The flow time is defined as the time corresponding to the minimum rate of change of axial strain during a creep test. The computational procedure for the flow time included in the first-article specifications includes three steps: (1) numeri- cal calculation of the creep rate; (2) smoothing of the creep rate data; and (3) identification of the point at which the minimum creep rate occurs as the flow time. The numerical calculation of the creep rate uses a simple finite difference calculation using data one sampling point ahead and one sampling point behind the point of interest. Smoothing of the creep rate is done using a five-point moving average filer. Finally, the flow time is reported as the time at which the minimum value of the smoothed creep rate occurs. If there is no minimum, then the flow time is reported as being greater than or equal to the length of the test. If more than one point shares the minimum creep rate, the first such minimum is reported as the flow time. Details of the calculations are included in the first-article specifications in Appendix A. 2.1.8.2 Flow Number The flow number is defined as the number of load cycles corresponding to the minimum rate of change of permanent axial strain during a repeated load test. The computational procedure for the flow number included in the first-article specifications is the same as that for the flow time, except the derivatives are taken with respect to the number of load cycles instead of with respect to time. The same smoothing and reporting as described above for the flow time is used with the flow number. 2.1.8.3 Dynamic Modulus The dynamic modulus test data is the most complex data to analyze. The material properties computed from this test are the dynamic modulus, |E*|, which is a measure of the material stiffness, and the phase angle, δ, which is a measure of the vis- cous properties of the material. Various methods are avail- 12 able for reducing data collected during the dynamic modulus test and computing the material properties. These include peak search algorithms, Fourier transform, and regression. The approach included in the first-article specifications is the regression approach (11). Regression was used because it is a relatively simple, direct approach that most engineers and technicians in the paving industry can understand and apply. This approach also lends itself to the computation of data qual- ity measures, which can be used in evaluating the reliability of test data. A step-by-step description of the regression approach is included in the first-article specifications in Appendix A. This approach is general and can be adapted to any number of specimen deformation transducers. In addition to the dynamic modulus and phase angle, the computations described in the first-article specification include four measures of data quality. As experience is gained with the dynamic modulus test, these data quality measures will be use- ful to engineers and test technicians in identifying the relia- bility of test data. The four data quality measures are described briefly below. 1. Standard Error of the Load. The standard error of the load is a measure of how well the applied loading approximates a sine wave. It is calculated from the dif- ference between the measured data and the best-fit sine wave using Equation 6. High values of standard error indicate poor sinusoidal loading, and such data should not be used for computing viscoelastic material prop- erties. The first-article specification limits the standard error of the load to 5 percent. (6) where se(P) = Standard error of the applied load xi = Measured load at point i xˆi = Predicted load at point i from the best fit sinusoid xˆo = Amplitude of the best fit sinusoid n = Total number of data points collected dur- ing test. 2. Standard Error of the Deformations. The standard error of the deformations is a measure of how well the specimen response approximates a sine wave. It is cal- culated individually for each deformation sensor using Equation 7, then averaged over the number of defor- mation sensors. High values of deformation standard error indicate poor specimen response or high amounts of signal noise. Such data should not be used for the computation of viscoelastic material properties. se P x x n x i i i n o ( ) ( ˆ ) % ˆ = − −  = ∑ 2 1 1 100

(7) where se(Yj) = Standard error for transducer j, % |Yj*| = Amplitude for transducer j ˆYji = Predicted response for transducer j at point i Yji = Measured response for transducer j at point i. 3. Uniformity Coefficient for the Deformation Ampli- tude. This parameter is a measure of the difference in deformations on the same specimen by the various deformation transducers. It is defined as the coefficient of variation of the deformation amplitudes measured by the individual transducers. Large differences in defor- mation measured on the same sample indicate suspect data. These differences may be caused by poor trans- ducer mounting, poor specimen end preparation, a faulty transducer, and inherent variability in the specimen. 4. Uniformity Coefficient for the Phase Angle. This parameter is a measure of the difference in phase angle measured on the same specimen by the various defor- mation transducers. It is defined as the standard devia- tion of the phase angles measured by the individual transducers. Large differences in phase angle measured on the same specimen indicate suspect data. These dif- ferences may be caused by poor transducer mounting, poor specimen end preparation, a faulty transducer, and inherent variability in the specimen. Values of the data quality measures for good and poor quality data were not available before the laboratory testing phase of Project 9-29. The specification requirement that the machines be capable of operating with a load standard error of 5 percent or less was based on discussions with equipment se Y Y Y n Y j ji ji i n j ( ) ˆ % * = −( ) −       = ∑ 2 1 1 100 13 manufacturers. An analysis of the data quality measures col- lected during the laboratory evaluation is included in Section 2.3 of this chapter. 2.1.9 Calibration The first-article specifications address both static and dynamic calibration of the simple performance test system. The specifications require the device to have a calibration mode and clearly marked access points for calibration by third-party services. The static calibration requirements and methods are summarized in Table 8. Neither AASHTO nor ASTM has a standard method for verification of temperature calibration. The first-article specifications include a method that uses a National Institutes of Standards and Technology (NIST) traceable thermal detector to compare temperatures measured with this device and the simple performance test system temperature sensor at six temperatures over the range of the environmental chamber. Dynamic calibration is also not addressed by current AASHTO or ASTM methods. The first-article specifications require a verification of dynamic performance to be performed after static calibration is complete. The approach is similar to that described in the laboratory start-up procedure for the Long Term Pavement Performance (LTPP) resilient modu- lus testing (12). This verification involves loading an elastic device, such as a proving ring, under static and dynamic con- ditions and recording loads and deformations. The first-article specifications require static and dynamic measurements to agree to within 2 percent and the phase angle between load and displacements be less than 1 degree. 2.1.10 Documentation and Warranty The first-article specifications include requirements for online and hard copy documentation. The specifications also include a 1-year warranty period for the first-article devices. TABLE 8 Static calibration requirements

2.2 FIRST-ARTICLE EQUIPMENT 2.2.1 Background A key component of NCHRP Project 9-29 was the involve- ment of equipment manufacturers early in the specification development process. Equipment manufacturers were asked to review and comment on the draft specifications developed by the research team. Thirteen equipment manufacturers were provided the draft specifications developed during Phase I of the project. Ten of those manufacturers also attended the manufacturers’ workshop held on July 30, 2001. At this work- shop, the draft equipment specifications were reviewed in detail with the manufacturers. Comments from the manu- facturers were then incorporated in the First-Article Equip- ment Specification for the Simple Performance Test System. Table 9 lists the manufacturers who participated in the devel- opment of the first-article equipment specifications. 2.2.2 Request For Proposals A request for proposals (RFP) for the Simple Performance Test System was issued on November 12, 2001, to the 13 man- ufacturers listed in Table 9. The RFP required the manufac- turers to provide information on their capabilities, a detailed description of their proposed simple performance test sys- tem, and information on pricing for the first-article and sub- sequent production units. The following four manufacturers submitted proposals in response to the RFP: 14 1. EnduraTec 2. Instron Corporation 3. Interlaken Technology Corporation 4. Shedworks, Inc. MTS Systems Corporation, and James Cox and Sons, Inc., responded that they could provide equipment to meet the specification, but declined to propose. Both companies indi- cated that they intended to monitor the market and might be interested in providing equipment in the future. 2.2.3 Proposal Evaluation Process This section summarizes the evaluation of the four pro- posals for the Simple Performance Test System. The four proposals were evaluated by five senior members of the research team. The evaluation panel and their particular areas of expertise relevant to the evaluation are summarized in Table 10. The proposals were evaluated independently by each panel member based on the criteria presented in Table 11. The first criteria addressed the first-article specification requirements. The RFP requested that the manufacturers describe their equipment in sufficient detail to document that the proposed equipment met the requirements of the first-article specifica- tion. The second criteria addressed advantages of the equip- ment in the areas of user-friendliness and reliability. The RFP informed the manufacturers that the primary use of the equip- TABLE 9 Equipment manufacturers

ment would be routine specification compliance testing by technicians in state highway agency, hot-mix producer, and consultant testing laboratories. The third criteria addressed the cost of the proposed system. As part of their proposal, the manufacturers were asked to provide a firm fixed price for the first-article equipment and estimates of the cost of future production units. Finally, the fourth criteria addressed the capabilities and experience of the manufacturer. Of particu- lar interest was documentation of the results of past prototype development projects. Each panel member was also asked to provide written comments and a recommendation of the two manufacturers who should be awarded contracts for the first-article equip- ment. A meeting to reconcile differences between evaluation panel ratings was planned, but was not required because of the consistency of the initial ratings. 2.2.4 Proposal Evaluation Results Three of the four manufacturers: EnduraTec, Interlaken, and Shedworks, proposed relatively compact equipment specifically designed to perform the three simple performance tests. The Instron proposal was essentially an assembly of standard and optional components for a general purpose load frame with customized software to perform the three simple performance tests. The similarities in the designs provided by EnduraTec, Interlaken, and Shedworks were striking. All proposed fairly compact bottom-loading equipment with a chamber that is both a pressure cell and temperature control chamber. They also proposed automatic or semi-automatic methods for opening the vessel for insertion of the test spec- imen. The primary differences in the designs were the load- ing system, the specimen deformation measuring systems, and the temperature control system. The results of the evaluation are summarized in Table 12. Although there are significant differences between evalua- tors in scores on individual criteria, there was overall agree- ment on the two highest ranking proposals. Additionally, the 15 evaluators unanimously recommended that the Interlaken and Shedworks designs be selected for the first-article equip- ment. These two proposals were rated near the highest by all evaluators on all of the evaluation criteria. The sections below summarize key elements of the equipment proposed by each manufacturer. 2.2.4.1 Instron The Instron approach was not well received by four of the five evaluators. The proposed approach offered some advan- tages for laboratories interested in using the equipment to perform a variety of tests in addition to the simple perfor- mance tests. The evaluators unanimously agreed that evalu- ation of this type of equipment would not provide significant benefit to future efforts to implement the simple performance tests. Although the cost of production units of this equip- ment, estimated by Instron at $65,000 to $70,000, is high for the market envisioned for the simple performance test, it sug- gests that equipment capable of performing the dynamic modulus master curves proposed by the Project 1-37A team for structural design may be available for approximately $100,000. A wider temperature range and higher load capac- ity than specified for the simple performance test are needed to perform the dynamic modulus test at the lower tempera- tures required for construction of master curves for structural design. 2.2.4.2 EnduraTec EnduraTec proposed a very innovative design based on the linear motor technology that they have developed in cooperation with Bose Corporation. The linear motor is an electromagnet that operates using standard electrical power available in all laboratories and has the potential to be very reliable and require minimal maintenance. Apparently this technology is not capable of providing both static and dynamic loads; therefore, the design included a pneumatic actuator and TABLE 10 Evaluation panel and expertise TABLE 11 Evaluation criteria

a load-sharing mechanism to provide the haversine loading required by the dynamic modulus and flow number tests and the static loading required by the flow time test. Three of the evaluators were concerned about whether this system could be controlled within the tolerances specified, and EnduraTec pro- vided no data to support their claim that it could. EnduraTec also proposed an innovative temperature control system. The system uses heating bands to provide heat, a solid-state ther- moelectric cooling (Peltier) device for cooling, and an internal circulating fan. Again, no data were provided to support the claim that the system could reach the specified temperature in the 3-minute time limit and control temperatures within the tolerances specified. The combined pressure vessel and envi- ronmental chamber has a locking flange to facilitate specimen insertion and a counter balance to enable the chamber to be easily lifted. The specimen deformation system consisted of two strain gauge extensometers with unique spring-loaded holders to keep them in contact with the specimen. This specimen deformation system combined with the counter- balanced, locking flange vessel has the potential to greatly 16 simplify specimen installation. Control of the entire system is provided through EnduraTec’s standard WinTest Control system, programmed for the three applications. EnduraTec has had limited experience with asphalt testing equipment. They have attempted to market the Field Shear Test device and redesigned equipment in support of NCHRP Project 9-18. The EnduraTec proposal received low ratings primarily because of concerns about the loading and temperature con- trol systems and the overall cost of the equipment. The cost of the first-article, at $89,480, was well above the Project 9-29 budget. EnduraTec’s estimated cost of production units at $55,000 to $63,000, depending on the market size, is somewhat above the Project 9-29 target of $50,000. 2.2.4.3 Interlaken Interlaken proposed a hydraulic-powered device that is a variation on two of their standard product lines: the Univer- sal Soils and Asphalt Test System and the ServoPress, which Maximum Score 300 300 200 200 1000 M an uf ac tu re r Ev al ua to r A bi lit y of P ro po se d Eq ui pm en t t o M ee t t he Sp ec ifi ca tio n R eq ui re m en ts A dv an ta ge s of th e Pr op os ed E qu ip m en t Co st o f F irs t A rti cl e Eq ui pm en t a nd Pr od uc tio n Un its Ca pa bi lit ie s an d Ex pe rie nc e of th e M an uf ac tu re r To ta l R ec om m ed ed M an uf ac tu re rs EnduraTec Instron Interlaken Shedworks Bonaquist 150 270 100 160 680 Christensen 201 219 106 152 678 Knechtel 240 150 80 80 550 Jack 300 150 60 100 610 Stump 240 240 60 200 740 Average 226 206 81 138 652 Bonaquist 240 60 100 100 500 Christensen 285 219 88 152 744 Knechtel 150 30 20 60 260 Jack 150 0 100 100 350 Stump 120 120 80 160 480 Average 189 86 78 114 467 Bonaquist 180 270 160 140 750 X Christensen 252 228 142 156 778 X Knechtel 240 240 180 100 760 X Jack 300 150 180 200 830 X Stump 180 300 160 200 840 X Average 230 238 164 159 792 Bonaquist 240 240 200 180 860 X Christensen 225 210 200 144 779 X Knechtel 240 210 180 180 810 X Jack 300 150 200 200 850 X Stump 300 240 200 200 940 X Average 261 210 196 181 848 TABLE 12 Summary of evaluation

is used for quality control in the metal-forming industry. The Interlaken Simple Performance Test System, shown in Fig- ure 6, is a small self-contained unit that includes the actuator and testing fixture, hydraulic supply, system control electron- ics, and computer interface in a small bench that is on cast- ers to provide mobility in the laboratory. The adaptation of proven reliable technology to the Simple Performance Test System is one of the reasons that the Interlaken proposal received high scores from all of the evaluators. In Inter- laken’s design, the combined pressure and temperature enclo- sure is automated using pneumatic cylinders to raise and lower the enclosure and latches to hold it in place during testing. Heating of the chamber is provided by an electrical resistance heater inside the enclosure. Cooling uses a heat exchanger inside the chamber that is cooled by a vortex chiller mounted outside of the enclosure. A small blower is included to pro- vide circulation within the chamber. The use of an automated, combined pressure and temperature vessel greatly simplifies equipment operation. Interlaken also proposed an automated system for mea- suring deformations in the dynamic modulus test. The sys- tem, shown in Figure 7, uses LVDTs that are mounted on guide brackets, and the brackets are pressed against the spec- imen by small pneumatic actuators. This specimen deforma- tion system combined with the automated enclosure greatly simplifies specimen installation. Control of the test system is provided through Interlaken’s digital controller and their UniTest software programmed for the three specific applications. An interesting aspect of the Interlaken software is the ability to provide access levels to 17 different users. Using this, a technician may be given only the ability to run an application. The laboratory manager, on the other hand, would have greater access and might be able to modify the control or data analysis. The Interlaken system was selected for the adaptation of proven technology, user considerations in the design, the experience building asphalt testing equipment, and cost. The cost of the first-article at $49,900 was at the NCHRP Project 9-29 target of $50,000. It is interesting that Interlaken’s estimated production unit costs remain within 10 percent of the first-article costs even for a very large number of units. This may be the result of cost savings already included in the first-article from the use of the same platform for the Simple Performance Test System and other standard prod- uct lines. 2.2.4.4 Shedworks Shedworks proposed to provide a user-friendly system that is an improvement on the equipment used at the Arizona State University in Project 9-19. The Shedworks Simple Perfor- mance Test System is a hydraulic powered unit that is a vari- ation of their compact, automated rapid triaxial test equip- ment. The unit, shown in Figure 8, includes two separate parts: a hydraulic power supply and the simple performance test equipment. The system is controlled by Industrial Process Controls’ (IPC’s) control and data acquisition system (CDAS2) that has already been programmed for the three simple per- formance test applications. Figure 6. Overview of Interlaken simple performance test system.

18 The Shedworks Simple Performance Test System includes a combined pressure and temperature chamber that is auto- matically lifted and lowered to facilitate installation of the specimens. The unique concept proposed by Shedworks is to control the temperature inside the chamber by supplying air at the required temperature. The system uses a refrigerated dryer to produce cool dry air that is then heated to the desired temperature with a process heater controlled by a sensor inside the cell. The use of thermally conditioned air for temperature control is an interesting concept that simplifies the equipment operation. The Shedworks specimen deformation measuring system, shown in Figure 9, is an improvement on the system used in NCHRP Project 9-19. It uses three LVDTs spaced equally around the circumference of the specimen. The LVDTs are held by a unique clip holder that allows rapid attachment of the LVDTs. Figure 9 shows an LVDT attached to the speci- men. The holder attaches to small disks that are glued to the specimen prior to conditioning them to the test temperature. The Shedworks Simple Performance Tester includes the device shown in Figure 10 to accurately position the glue-on disks on the specimen. The Shedworks system was selected for their adaptation of proven technology, user considerations in their design, and their experience building asphalt testing equipment, particu- larly that used by the Arizona State University in Project 9-19 and cost. The cost of the first-article at $39,000 was well below the NCHRP Project 9-19 budget. The estimated cost Figure 7. Interlaken dynamic modulus test instrumentation. Figure 8. Shedworks simple performance test system.

of production units at $25,000 is also well below the NCHRP Project 9-19 target of $50,000. 2.3 FIRST-ARTICLE EQUIPMENT EVALUATION The evaluation of the first-article equipment had several objectives. The first was to assess the specific equipment procured in Phase II of the project, make recommendations concerning the acceptability of this equipment to perform the specified testing, and evaluate the functionality of the equip- ment for use in routine laboratory testing. The second objec- tive was to evaluate the repeatability and reproducibility of material properties measured with equipment manufactured to the same specification by two vendors and to compare that with data from two laboratories. The third objective was to identify possible revisions to the first-article equipment spec- ification that will enhance the functionality of future equip- ment or reduce variability in measured material properties. Finally, the fourth objective was to identify possible revisions to the Project 9-19 Draft Test Protocols to simplify testing and reduce variability in measured material properties. To accom- plish these objectives, the first-article equipment evaluation included two major components: specification compliance testing and mixture testing. 19 The specification compliance tests were developed to document that the equipment meets the requirements of the first-article specifications. The tests were included in the first-article specification and successful completion of these tests was a requirement of the purchase orders issued to Interlaken and Shedworks. The objective of the mixture com- ponent of the first-article evaluation was to evaluate the repeatability and reproducibility of material properties mea- sured with equipment manufactured to the same specifica- tion by two vendors. Given that this component involved the preparation and testing of a large number of specimens, the functionality, and to a certain degree, the durability of the equipment was also evaluated. Finally, this component of the first-article evaluation provided the opportunity for the evaluation of the Project 9-19 test protocols by practicing technicians. 2.3.1 Specification Compliance Testing Authorization to proceed with fabrication of the Simple Performance Test System was given to Interlaken and Shed- works on January 18, 2002. Both systems were completed and delivered within the specified time frame. The Shed- works device was completed first and delivered to Advanced Asphalt Technologies’ (AAT’s) laboratory on July 10. The Interlaken device was delivered to the FHWA Turner-Fairbank Highway Research Center on July 22. Upon delivery, repre- sentatives of the manufacturers set up the equipment and par- ticipated in the specification compliance testing, which was designed to verify that the equipment met the specification requirements. Table 13 summarizes the items included in the specification compliance testing. The specification compliance testing for the Shedworks device was performed from July 15 through July 19. The equipment was found to be in compliance with the specifica- tion. Some minor software issues were noted. Shedworks pro- vided revised software addressing the software issues before the start of the evaluation testing in November, 2002. The specification compliance testing for the Interlaken device was initially performed from July 23 through July 26. The equipment failed several of the specification compli- ance tests. Table 14 presents a summary of the deficiencies initially found in the Interlaken equipment. The research team worked with representatives of Interlaken throughout August and early September to resolve these deficiencies. Representatives from Interlaken visited the Turner-Fairbank Highway Research Center twice during this period to make substantial changes to the hardware and software. Interlaken completed resolution of the deficiencies on September 13, 2002, and on September 16 and 17, the research team veri- fied that the equipment met all of the specification compli- ance tests. Figure 9. Shedworks dynamic modulus test instrumentation.

2.3.2 Mixture Testing 2.3.2.1 Experimental Design Although the first-article simple performance devices are capable of performing three tests (i.e., dynamic modulus, flow number, and flow time), only two of these tests were included in the mixture testing component of the evaluation because of budget constraints. The dynamic modulus and flow number were the tests selected for evaluation because these were the two tests for which criteria differentiating between good and poor performance were being developed in Project 9-19. Research in Project 9-19 found a good cor- relation between flow number and flow time, allowing flow time to be used as a surrogate test for flow number, but the criteria differentiating between good and poor performance will be based on the flow number test and the performance of in-service sections. Tables 15 and 16 present the experimental design for the dynamic modulus and flow number tests. Data for the two simple performance test devices were collected in two labo- ratories (AAT and FHWA) on two mixtures (9.5 mm and 19.0 mm). Eight independent tests were included in each cell to provide sufficient replication to evaluate differences in means and differences in variances between devices, labora- tories, and testing conditions. The dynamic modulus tests were 20 conducted for three conditions selected to exercise the range of the equipment capabilities: • Unconfined dynamic modulus at 25 °C, a representative condition for evaluating mixtures for fatigue cracking potential; • Unconfined dynamic modulus at 45 °C, a representative condition for evaluating mixtures for rutting potential; and • Confined dynamic modulus at 45 °C, a representative condition for possibly evaluating open- or gap-graded mixtures for rutting potential. The flow number was evaluated only at 45 °C for unconfined and confined conditions. The levels of confinement and deviatoric stress were selected to provide a relatively short test, fewer than 1000 cycles, and a relatively long test, greater than 5000 cycles. 2.3.2.2 Mixtures Two mixtures that exhibited different levels of variability in mechanical properties when tested in NCHRP Project 9-18, Figure 10. Shedworks glue-on gauge point system.

“Field Shear Test for Hot-Mix Asphalt,” were used in the eval- uation testing. The first was a 9.5 mm mixture with low vari- ability, having a shear modulus coefficient of variation of approximately 5 percent when tested in NCHRP Project 9-18 (13). The second was a 19.0 mm mixture that had a shear mod- ulus coefficient of variation of approximately 17 percent (13). 21 Volumetric properties for the mixtures are provided in Table 17. Both are coarse-graded Superpave mixtures. The 9.5 mm mixture was made with limestone coarse and fine aggregates. Granite aggregates were used in the 19.0 mm mixtures. Both mixtures were made with the same PG 64-22 binder. AASHTO M320 properties for the binder are summarized in Table 18. TABLE 13 Summary of specification compliance tests

2.3.2.3 Laboratory Methods Preparation and testing of the simple performance tests specimens was performed in accordance with the Project 9-19 test protocols (1). Appendix B provides a detailed descrip- tion of the laboratory methods. The simple performance test specimens were prepared to a target air void content of 4.0 percent. First 150-mm diam- 22 eter by 165-mm high gyratory specimens were prepared to air void contents of approximately 5.5 percent. From these, 100-mm diameter by 150-mm high specimens were cored and sawed using the portable core drilling machine shown in Fig- ure 11 and the double-bladed saw shown in Figure 12. All cor- ing and sawing was done using water to cool the cutting tools. After all cutting was complete, the bulk specific gravity of the finished specimen was determined in accordance with TABLE 14 Summary of initial specification test deficiencies for the Interlaken equipment TABLE 15 Experimental design for dynamic modulus testing

AASHTO T166 by first measuring the immersed mass, then the saturated surface dry mass, and finally the dry mass. The cores were measured for compliance with the NCHRP Project 9-29 specimen tolerances, which are summarized in Table 19. The dynamic modulus and flow number tests were per- formed with the simple performance test devices in accor- dance with the Project 9-19 test protocols (1). Test specimens were conditioned in a separate environmental chamber prior to testing. Dummy specimens with thermocouples were used to ensure that the test specimens were within the specified 0.5 °C tolerance of the target test temperature. The test chamber of the simple performance test device was also equilibrated to the target testing temperature. Once the specimens and the test chamber reached the target temperature, the specimens were removed from the separate environmental chamber, placed in the test chamber, and instrumented if required. The 23 test chamber was then closed and allowed to equilibrate to the test temperature before the testing began. The three dynamic modulus tests, 25 °C unconfined, 45 °C confined, and 45 °C unconfined, were performed on the same test specimen. For each condition, dynamic moduli and phase angles were measured at frequencies of 25, 10, 5, 1, 0.5, and 0.1 Hz. Stress levels were varied automatically by the sim- ple performance testers to achieve a target strain level of 100 µstrain. A confining pressure of 138 kPa was used in the confined testing. Separate test specimens were used for each of the flow time tests. Table 20 summarizes the confining and deviatoric stresses used in the flow number testing for the two mixtures. The evaluation testing program required fabrication and testing of 192 specimens. Sample fabrication and testing were split into two phases, as shown in Table 21. In the first phase, the Interlaken equipment was operated in the FHWA laboratory and the Shedworks equipment was operated in AAT’s laboratory. In the second phase, the location of the equipment was switched. Each phase was divided into two blocks, and all of the testing for a given block in both labo- ratories was completed before the next block began. To allow reasonable productivity during specimen fabrication, the over- height gyratory specimens were fabricated on a regular sched- ule of four specimens per day. To minimize aging of the test specimens, the simple performance test specimens were sawed and cored from the over-height gyratory specimens when TABLE 16 Experimental design for flow number testing TABLE 17 Volumetric properties of evaluation mixtures

needed. All of the simple performance test specimens for a specific mixture for a block were cored, sawed, and measured at the same time. They were then distributed to the two labo- ratories based on their air void contents to obtain approxi- mately the same average and range of air void contents. 2.3.3 Statistical Analysis 2.3.3.1 General This section presents key findings from the statistical analysis of the mixture testing component of the first-article evaluation. The data collected during the first-article evalua- 24 tion are presented in Appendix C. The primary objectives of the statistical analysis were as follows: 1. To evaluate the general overall quality and reasonable- ness of the data generated with the two devices under different conditions; 2. To evaluate the variability in the data and what differ- ences in variability occur with different devices and test conditions; and 3. To evaluate significant differences in the mean response produced using the devices under different conditions. The dynamic modulus and flow number experimental designs presented earlier represent analysis of variance exper- iments with four factors: • Device (Two levels: Shedworks (IPC), Interlaken (ITC)); • Laboratory (Two levels: AAT and Federal Highway Administration (FHWA)); • Mixture (Two levels: 9.5-mm and 19-mm); and • Test conditions (Dynamic modulus test, three levels: 25 °C unconfined, 45 °C unconfined, and 45 °C confined) (Flow number test, two levels: 45 °C unconfined and 45 °C confined) Because it is well known that changes in temperature and confinement will produce substantial changes in the mechan- ical properties of asphalt concrete and because changes in temperature and confinement probably produce differences in variance that might render a statistical analysis invalid, the analysis was performed separately for each test condition. Thus, the experiment design for practical purposes involved a full 23 factorial, that is, an analysis of variance including three factors (i.e., device, laboratory, and mixture), each at two levels. Because one of the primary purposes of these experi- ments was to evaluate and compare the variances among the different factors, a large number of replicates was tested— eight for each cell. Thus each of the experiments included 64 independent measurements. As will be seen in the following discussion, there were many cases where statistically significant differences in standard deviation occurred, depending on the specific test conditions. For this reason, analysis of variance techniques were not used in the analysis. This did not severely handi- cap the results, because the objectives of the analysis could just as easily be achieved by a combination of simple com- TABLE 18 Binder properties for evaluation mixtures Figure 11. Portable core drilling machine and stand.

parisons between standard deviations and mean response values. 2.3.3.2 Dynamic Modulus Each test in the dynamic modulus experiment included a frequency sweep using six frequencies: 25, 10, 5, 1, 0.5, and 0.1 Hz. Given that the responses at different frequencies tend to be similar and closely related, a rigorous statistical analy- 25 sis was not performed at each frequency. Only the data from the 10, 1, and 0.1 Hz frequencies were included in the analy- sis. The analysis proceeded in the following order. First, var- ious plots were constructed to observe general trends in the modulus and phase angle data. Second, a detailed analysis of the equality of variances between the various cells of the experiment was performed. This second step was critical to the selection of appropriate methods to evaluate differences in mean response. The third step was an analysis of differ- ences in mean response for the two devices and laboratories. Figure 12. Double-bladed saw. TABLE 19 Project 9-29 specimen dimension tolerances TABLE 20 Flow number test conditions

The final step was an analysis of quality statistics from the tests to determine overall levels of variability for the dynamic mod- ulus test and to recommend limits for the quality indicators to be included in the test protocol. The sections that follow pre- sent and discuss pertinent findings from these four analyses. 2.3.3.2.1 General Trends. Figures 13 and 14 show the relationship between dynamic modulus (|E*|) data generated using the two devices and at the two laboratories, respec- tively. Note that at intermediate- to high-modulus values, the two devices appear to agree closely, although at lower mod- ulus values, the Interlaken device seems to produce higher values for |E*|. Modulus data generated at AAT and FHWA appear to be similar, regardless of the mixture stiffness (see 26 Figure 14). Given that both devices were calibrated to the same standards before testing, and the same testing protocol was used in both laboratories, the likely cause of the differ- ences in |E*| shown in Figure 13 is the specimen deformation measuring system. Recall, the Interlaken device has an auto- mated extensometer system that uses air actuators to hold the deformation measuring system against the specimen. The Shedworks device uses a refined glued-on gauge point sys- tem similar to that used in the original Project 9-19 research. As discussed later, additional statistical analyses were per- formed to determine if the discrepancy shown in Figure 13 is statistically significant. Figure 15 shows a comparison of phase angle values gen- erated using the two simple performance test devices. Note TABLE 21 Evaluation testing program Figure 13. Comparison of dynamic modulus values generated using the Interlaken (ITC) and Shedworks (IPC) simple performance test devices (dashed line represents fit, solid line equality).

that, as with modulus, the relationship appears to deviate from equality, though the statistical significance of the deviation cannot be judged from this figure. In general, the Interlaken device appears to generate somewhat lower phase angles than the Shedworks device at high phase angles, corresponding to low modulus values. This difference is consistent with that observed for |E*|, where the Interlaken device produced higher modulus values than the Shedworks device for low mixture stiffness values. Figure 16 shows 95-percent confidence limits for the stan- dard deviation of |E*| for the 25 °C tests for 24 combinations of laboratory, device, mixture, and frequency. These are not joint confidence limits, but single confidence limits calcu- lated using n = 7 degrees of freedom and α = 0.05—that is assuming a 5-percent risk of failure to capture the true value for the standard deviation (14). It is clear in this figure that the standard deviation decreases significantly with decreas- ing frequency. However, it should be remembered that |E*| 27 also decreases with frequency, so that the variability relative to the modulus value is probably relatively constant. To eval- uate the change in variance relative to modulus value, the confidence limits were converted to confidence limits in coef- ficient of variation (C.V.), by expressing them as a percent- age of the measured |E*| value, and plotted in Figure 17. The variability expressed in these terms appears independent of frequency. It is difficult to evaluate other aspects of the changes in variability with test conditions from these figures, though it appears that the overall level of variability is rela- tively low for measurements of mixture modulus. In summary, the general trends in the dynamic modulus data show that the modulus and phase angle data generated by the two devices at the two laboratories appear reasonable and are in general agreement. The Interlaken device appears to produce higher |E*| values and lower phase angles than the Shedworks device for low stiffness conditions. The overall variability of the dynamic modulus data produced with both Figure 14. Comparison of complex modulus values generated at FHWA and at AAT using the two simple performance test devices (dashed line represents fit, solid line equality). Figure 15. Comparison of phase angle values generated by the Interlaken (ITC) and Shedworks (IPC) simple performance test devices (dashed line represents fit, solid line equality).

simple performance devices is reasonable, with coefficients of variation for various conditions ranging from 5 to 15 percent. 2.3.3.2.2 Detailed Analysis of Variability. Experiments of the type performed in this project are often analyzed using analysis of variance techniques. One of the assumptions in 28 analysis of variance is that the variances for the different fac- tors are the same (14). To evaluate this assumption and poten- tial differences in variability between the two first-article devices, a detailed analysis of the equality of the variances in the cells of the experiment was performed. Two statistical tests were used to evaluate the equality of variances. In the Figure 16. 95% confidence limits for |E*| standard deviation at 25 °C, unconfined. Figure 17. 95% confidence limits for |E*| coefficient of variation at 25 °C, unconfined.

cases where two variances were compared, such as compar- ing the variability of the Interlaken device to that of the Shed- works device, an F-test was used. The F-test was performed as follows (14): H0: σ1 ≥ σ2 Ha: σ1 < σ2 If F(α; n1 − 1, n2 − 1) ≤ s12/s22, conclude H0, otherwise con- clude Ha where F is the value of the F-test statistic for the specified value of α and degrees of freedom, ni is the number of obser- vations, si is the sample standard deviation, and α is the prob- ability of incorrectly concluding that one standard deviation is larger than the other. In other cases, an analysis of the equivalence of more than two variances was desired, an example being a check on the equality of variances for data collected in both laboratories with both devices for the 9.5 mm mixture. For these cases, all standard deviations can be compared simultaneously using the Hartley test (14): H0: σ1 = σ2 = σ3 = σ4 Ha: not all σi all equal If H(1 − α; r, n − 1) ≥ max(s i2 )/min(s i2 ), conclude H0, other- wise conclude Ha 29 where H is the Hartley test statistic, and r is the total number of sample standard deviations being compared. To evaluate the variability in the data generated at the two laboratories thoroughly using the two devices, both the Hartley test for equality of standard deviations and F-tests for com- paring standard deviations (between devices and between lab- oratories) were performed for combinations of major factors: temperature/confinement, frequency (10 Hz and 0.1 Hz), and mixture. The results for comparisons of the standard deviation of |E*| are summarized in Table 22. Values in boldface type are considered to show significant differences. These cases have a probability of incorrectly concluding that the standard deviation for one condition tested is greater than the others (α) less than or equal to 0.10. Out of the total of 12 cases, the Hartley test indicates that the standard deviations are not all equal in 3 cases. There does not appear to be any pattern to those situations exhibiting unequal standard deviations. Using the F-test to compare standard deviations between devices, in 4 out of the 12 cases the Interlaken device exhibits a larger value than the Shedworks device. In three cases, the reverse is true. In comparing the standard deviations between the two laboratories, in only one case does one of the labora- tories exhibit significantly greater variability than the other— for the 9.5-mm mixture at 45 °C (confined) and 0.1 Hz, data produced by the FHWA shows greater variability than that produced by AAT. The corresponding summary of comparisons of standard deviations for phase angle is shown in Table 23. In this case, the Hartley test shows unequal standard deviations in 5 of the 12 cases. In comparing devices, the Interlaken device exhibits greater standard deviation values in 7 of 12 cases, while the TABLE 22 Summary of statistical tests for comparison of modulus standard deviations

Shedworks device exhibits greater standard deviation values in 3 of 12 cases—all at 45 °C, confined. As with modulus, in only one case does one of the laboratories exhibit a greater standard deviation than the other, and it is again FHWA. The primary reason for including both a 9.5-mm aggregate gradation and a 19-mm aggregate gradation was that the vari- ability for these mixtures was expected to be different. In general, most paving engineers and technicians believe that 30 mixtures with larger nominal maximum aggregate sizes are more difficult to work with and will exhibit more variability in the results of mechanical property tests. Table 24 is a sum- mary of comparisons between standard deviations for mix- tures made with the two aggregate types. For modulus val- ues, the 19-mm mixture appears to exhibit greater variability, with a larger standard deviation in 10 of 12 cases. However, the modulus values for the 19-mm mixture are somewhat TABLE 23 Summary of statistical tests for comparison of phase angle variance values TABLE 24 Summary of statistical tests for comparison of modulus and phase angle variability between aggregate types

higher than for the 9.5-mm mixture; and, for this type of mea- surement, variability tends to increase with higher values of modulus. Therefore, coefficient of variation (C.V.) is a bet- ter indicator of variability. Unfortunately, strict statistical tests cannot be constructed using C.V. However, an approximate test can be constructed using C.V. in place of standard devia- tion. The results of the approximate F-tests using C.V. is also included in Table 24. In this case, the 19-mm mixture exhibits greater variability in only 4 of 12 cases, and the 9.5-mm shows greater variability in 1 case. It appears that, once adjusted for differences in mean response, the variability in modulus val- ues for the two mixtures is similar. The results for phase angle agree better with the results of the approximate C.V. F-tests. In 2 of 12 cases, the 19-mm mixture does exhibit greater vari- ability, while in 2 of 12 cases, the 9.5-mm mixture exhibits greater variability. It appears that the two mixtures, in general, exhibited similar levels of variability. Care should be taken in generalizing these results, as this series of tests involved only two different aggregates. It is also possible that in other situ- ations, with less careful control over conditions and specimen preparation, the 19-mm mixture would have shown greater variability than the 9.5-mm mixture. The detailed analysis of variability resulted in several per- tinent findings. These are summarized and discussed below: 1. There are significant differences in variance of the dynamic modulus and phase angle for the factor and treatment levels in the experiment; therefore, analysis of variance techniques cannot be used to analyze dif- ferences in mean response. 2. The variability in the dynamic modulus and phase angle data for the 9.5 mm and 19.0 mm mixtures is similar. This was an unexpected finding given that these mix- tures were selected because they exhibited large dif- ferences in variability during previous shear modulus testing. 3. The variability in data generated in the two laboratories is similar. This finding is probably the result of the pro- tocol used in the laboratory testing. First, all specimens were fabricated at AAT, then distributed to the two laboratories to have the same average and range of air voids. Second, the same temperature conditioning meth- ods were used in both laboratories. Specimens to be tested on a particular day were conditioned with a dummy specimen in a separate environmental cham- ber. Once the specimens and the device reached the test temperature, the specimen to be tested was removed and quickly inserted into the test chamber. The test chamber was closed and allowed to return to equilib- rium before testing proceeded. 4. The variability in data generated with the Interlaken device is higher than that generated with the Shed- works device for unconfined tests. However, the vari- ability for data generated with the Shedworks device is higher for confined tests. This finding can be rationally explained considering the configuration of the defor- 31 mation measuring systems. As discussed in a later sec- tion of this report dealing with functional characteris- tics, the Interlaken extensometer system requires further refinement. Often the system had to be re-seated sev- eral times to obtain acceptable contact with the speci- men. Thus, the higher variability for the Interlaken device for unconfined tests is probably due to slip or uncontrolled movement of the extensometer contact. In confined tests, the rubber membrane apparently pro- vides a better, more stable contact. For confined testing with the Shedworks device, the membrane is sand- wiched between the LVDT bracket and the glued-on contact point. The bracket is held in place with a screw that is tightened before confining pressure is applied. When confining pressure is applied, the membrane gets thinner as it stretches over the contact point, allowing the LVDT bracket to loosen. Greater variability in the test data, compared with that collected for unconfined testing when the membrane is not between the bracket and the contact would be expected. 2.3.3.2.3 Detailed Analysis of Mean Response. Analysis of variance is often used to analyze the significance of differ- ences in means in the type of experiment used here. However, as discussed previously, analysis of variance assumes equal- ity of means in the various cells being analyzed, which is clearly not the case for the dynamic modulus data. An approx- imate test of equality between two means can be made, even with somewhat unequal standard deviations. Because the sample sizes in all cases are equal (n = 8), the common stan- dard deviation for comparing means can be estimated using the following Equation 8 (14): (8) where s1 and s2 represent the standard deviations for the two measurements, and n1, n2 is the sample size for each case. For this experiment, n1 = n2 = 8; therefore, there are 14 degrees of freedom associated with the comparison of two mean val- ues. This is large enough to provide a very good estimate of the standard deviation, so that a normal distribution may be used in making the statistical test rather than the t-distribution used for small sample sizes. Then, the significance level, or chance of incorrectly concluding that the mean of one mea- surement is greater than the other, is given by Equation 9: (9) where N(α) represents the z-value at which there is only a chance, α, that it will be exceeded. This approximate test was conducted, comparing both mean values as determined using each device (averaged across laboratory) and as measured at each laboratory (averaged across device). The results for |E*| are shown graphically in Figure 18 for comparison between N E E sDIFF( ) * *α = −( )1 2 s s s n n DIFF = +  + 1 2 2 2 1 22 1 1

laboratories and Figure 19 for comparisons between devices. In these figures, only differences which are statistically sig- nificant (α = 0.10) are plotted. In comparing devices, 8 out of 12 comparisons were signif- icant (α = 0.10); in comparing laboratories, 10 out of 12 com- parisons were significant. Furthermore, it was found that the magnitude of the difference between laboratories appeared to depend on the mix type. For the 9.5-mm mixture, signifi- cantly higher modulus values were measured at the FHWA laboratory, while for the 19-mm mixture, higher values were measured at the AAT laboratory, though the difference in this case was not as great. The difference in |E*| as deter- mined using the two devices, on the other hand, appears to be independent of aggregate type. In this case, the Interlaken device measures higher modulus values, the difference becom- ing larger at lower modulus values. 32 The corresponding plot for comparison of mean phase angle values is shown in Figures 20 and 21. In these cases, the patterns are not as pronounced. For the comparison of phase angles measured by the two devices, 10 of 12 cases exhibited significant differences, with the ITC device gen- erally producing lower phase angle values, by as much as 6 degrees. For comparison of phase angles measured at the two laboratories, 8 of 12 showed significant differences; and, in each of these cases, the difference was less than 2 degrees. A second approach to comparing data from the two devices involves the use of regression in combination of con- fidence intervals. This provides a general evaluation of equal- ity, useful for evaluating bias in the data. The |E*| data were evaluated in this manner for three conditions: 25 °C uncon- fined, 45 °C unconfined, and 45 °C confined. In Figure 22, the log of |E*| at 25 °C as measured using the Interlaken Figure 18. Statistically significant (p ≥ 0.10) differences in modulus measurement for comparisons between devices, as a percent of mean value. Figure 19. Statistically significant (p ≥ 0.10) differences in modulus measurement for comparisons between laboratories, as a percent of mean value.

33 Figure 20. Statistically significant (p ≥ 0.10) differences in phase angle measurement for comparisons between devices. Figure 21. Statistically significant (p ≥ 0.10) differences in phase angle measurement for comparisons between laboratories. Figure 22. Regression of log |E*|/ ITC device as a function of log |E*|/ IPC device, 25 °C data only, with 95% confidence interval for the regression and line of equality.

device is shown as a function of log |E*| at 25 °C as measured using the Shedworks device. A log transformation was used to provide a better distribution of residuals. This plot includes the 95-percent confidence interval for the regression rela- tionship and the line of equality. In this case, the regression line appears to run parallel and quite close to the line of equality, but the confidence interval for the regression line does not quite capture the line of equality. Therefore, it appears that at 25 °C, modulus values measured using the Interlaken device are slightly greater than those measured using the Shedworks device. This bias, though consistent, is not large. Figure 23 shows the relationship between |E*| measured with the two devices for the 45 °C unconfined data only. In this case, the line of equality is not parallel to the regression line—instead, it falls below the regression line at low modu- lus levels and falls above it at high modulus values. As mod- ulus values decrease, the values measured by the Interlaken device become larger relative to those measured using the Shedworks device. Figure 24 shows the relationship between |E*| measured with the two devices at 45 °C, but for confined data. In this 34 case, the 95-percent confidence interval appears to capture the line of equality over most of the data range. As with the uncon- fined data, there is much more scatter than in the 25 °C uncon- fined data, which is probably due to the overall low modulus values and relatively low applied stress levels. Despite the higher noise, it appears that, in this case, the trends in the data are similar to those observed in the 25 °C unconfined data. To compare overall trends in modulus measurements for the two devices, Figure 25 was constructed, which shows the relationship between modulus values measured with both devices for all conditions, separately coded, and with indi- vidual regression lines (but no confidence intervals). It appears that the 25 °C unconfined and 45 °C confined data compare very well and follow a similar relationship, although the mod- ulus values at 25 °C were slightly higher for the Interlaken device. The unconfined data at 45 °C clearly follow a differ- ent relationship than the other two cases, with the Interlaken device producing higher modulus values at low modulus levels and lower values at higher overall modulus levels. The relationship between modulus values is made even clearer in Figure 26, which is a plot of the percent differ- Figure 23. Relationship between modulus as measured using ITC and IPC devices, 45 °C, unconfined data only, with 95% confidence interval for the regression and line of equality. Figure 24. Relationship between modulus as measured using ITC and IPC devices, 45 °C, confined data only, with 95% confidence interval for the regression and line of equality.

ence between modulus values as measured using the Shed- works and Interlaken devices. At modulus values above about 1,000 MPa, there is relatively little scatter in the data, and the Interlaken device produces slightly higher modulus val- ues compared with the Shedworks device. As the modulus decreases below 1,000 MPa, the scatter in the data becomes greater, and the difference between the two devices becomes greater. For the two data points below 200 MPa, the Inter- laken device produced values about 40 percent higher than those generated using the Shedworks device. The final regression plot in this series is shown in Figure 27, which is a plot comparing phase angle measurements made with the two devices at both temperatures. The variability in phase angle appears to be larger than the variability in |E*| measurements. At phase angles below about 28 degrees, the two devices appear to be in reasonable agreement. However, at higher phase angles, the Shedworks device produces slightly higher phase angle values. The major finding from the detailed analysis of the mean response is that dynamic moduli measured with the Inter- 35 laken device are higher than those measured with the Shed- works device for unconfined conditions. For confined condi- tions, dynamic moduli measured with the two devices are similar. These findings are also rationally explained by errors in the two measuring systems. As discussed previously in the detailed analysis of the equality of variances, the Inter- laken specimen-mounted deformation system probably has errors caused by movement at the point where the exten- someter contacts the specimen. Such errors result in lower measured strains and higher moduli. For confined condi- tions, the membrane appears to reduce these errors for the Interlaken device. For the Shedworks device, the membrane is sandwiched between the glued contact point and the LVDT bracket, producing a measuring system that also has relative movement errors. Thus, the net result of confinement is to reduce errors in the Interlaken measurement and increase errors in the Shedworks measurement, making the dynamic moduli for confined conditions the same. The lower phase angles for the Interlaken data are also consistent with this type of measurement error. Figure 25. Relationship between modulus as measured using ITC and IPC devices, all conditions, showing separate regression lines. Figure 26. Percent difference between modulus as measured using ITC and IPC devices, all conditions.

2.3.3.2.4 Analysis of Test Variability. The final statisti- cal analysis done was to assess the overall variability of the dynamic modulus measurements and the effect that the data quality indicators recommended in this project may have on reducing test variability. Table 25 summarizes pooled values for the coefficient of variation for dynamic modulus (|E*|) and for standard deviation for the phase angle. These values thus represent overall variability for the two devices, two lab- oratories, and two aggregates. The trends for modulus and phase angle are similar. The overall variability for the Shed- works device is slightly lower than for the Interlaken device, though the variability of the Interlaken device is better for the confined tests at 45 °C. The variability of data generated at the two laboratories appears to be similar. The overall vari- ability for the two aggregates also appears similar. It appears that the variability in test data at 45 °C is greater for uncon- fined than for confined data. The last column in Table 25 shows overall values for coefficient of variation and standard deviation. The overall coefficient of variation for modulus 36 is 13.0 percent and the standard deviation for phase angle is 1.73 degrees. These values are very similar to values reported from data collected in Project 9-19 as summarized in Table 26. Keep in mind that these values are for one replicate measure- ment only. Typically, three replicate measurements are made when measuring the modulus of asphalt concrete specimens. For n = 3 replicates, the coefficient of variation for average modulus would then be 13.0/√3 = 7.5%. The standard devi- ation for average phase angle for n = 3 replicates would be 1.0 degrees. This amount of variability appears to be quite good for mechanical measurements on asphalt concrete. The final part of this analysis is an evaluation of the quality indices that are part of the output of the dynamic modulus test with the first-article devices. These indices provide informa- tion concerning the accuracy of loading and response wave- forms, using statistical parameters such as standard errors. The following quality indicators are provided by the dynamic mod- ulus software included in the first-article devices: • Load standard error—this is the standard error of the load waveform compared with an ideal sine function of identical magnitude and phase lag. • Load drift—this is the amount of gradual, permanent change in the applied load during a test, in addition to the desired sinusoidal load. • Deformation standard error—the standard error between the actual deformation waveform and an ideal sine function of identical magnitude and phase lag. Figure 27. Regression of phase angle/ITC device as a function of phase angle/IPC device, with 95% confidence interval for the regression and line of equality (R2 = 73.2%, adj. for d.f.). TABLE 25 Pooled coefficient of variation for |E*| and standard deviation for phase angle at 10 Hz TABLE 26 Pooled coefficient of variation for |E*| and standard deviation for phase angle from studies involving a large number of dynamic modulus tests

• Deformation drift—the amount of gradual, permanent change in the deformation during a test, in addition to the sinusoidal component of deformation. • Deformation uniformity—this is the average difference between (or among) the amplitude of the deformation sig- nals expressed as a percentage of the mean deformation. If all signals are identical in amplitude, the deformation uniformity is zero. • Phase uniformity—the average difference between (or among) the deformation phase lags, expressed in degrees; a value of zero indicates that the deformations are com- pletely in phase. The initial evaluation of the quality indices involved deter- mining correlations between the various indices and the coef- ficient of variation of the modulus and standard deviation for the phase angle values. A high degree of correlation between a particular quality index and the modulus coefficient of vari- ation and/or the phase angle standard deviation would indi- cate that that indicator was potentially important in determin- ing the quality of the measurement. Low correlation, on the other hand, does not necessarily indicate that that quality index is not important. Low correlation suggests that it is either not important or, more likely, that the values for the index in this data set were low enough so that they did not have a substantial effect on the quality of the resulting data. For this particular set of data, R2 values between the qual- ity indices and the modulus coefficient of variation and the phase angle standard deviation were very low, ranging from 1 to 8 percent. It is believed that these low values suggest that the quality indicators were in the range where they did not cause significant problems in most of the data. Table 27 is a summary of the quality indices for the two devices, for load and deformation. The values for the indices have been bro- ken down by loading frequency, because the frequency had a significant effect on their magnitude. Shown in the table is the average value for each index at each of three frequencies, the standard deviation for the index, and the 95 percent con- fidence limit for each index. This confidence limit represents the value for the index which, in the long run, will only be exceeded one time in twenty, and so serves as a basis for establishing a limit for that index that can be used to identify questionable data. In general, the standard errors are lower for the Shedworks device compared with the Interlaken 37 device. The values at 0.10 Hz for the Interlaken device are not reported here, because it was found that they had been incorrectly calculated by the Interlaken software because the device was applying a loading slightly slower than 0.10 Hz. It was found that the standard errors for deformation are strongly dependent on the standard errors for load; the R2 value between these two indices was 81 percent. Although this might seem to suggest that only one of these indices need be specified, it is believed that both should be specified to ensure that devices and software produced in the future main- tain the needed quality in loading and measurement. Another trend in these quality indices is that the best quality data (low- est index values) are produced at 1 Hz, with poorer data at both 10 and 0.1 Hz. This might be the result of the devices having optimal performance characteristics at 1 Hz, or it might be the result of the manufacturers’ tuning process. As discussed previously, the Interlaken deformation mea- surement system, although innovative, easy to use, and quite promising, seems to exhibit some bias compared with the Shedworks system because of movement of the deformation transducers relative to the specimen. This should be expected to increase standard errors in deformation also. In examining Table 27, it is clear that the deformation standard errors for the Interlaken device are substantially larger than those for the Shedworks device. These values should, therefore, be interpreted with caution and should be disregarded in deter- mining preliminary limits for quality indices. After eliminat- ing these values from Table 27, it would appear that a rea- sonable general limit for load and deformation standard error would be 10 percent. Most test data would pass this limit. If the device tuning can be improved over the full range of fre- quencies, a lower limit of 7 percent can probably be applied. Table 28 is a summary of drift and uniformity coefficients for load and deformation for the two first-article devices. The load drift values are quite low, suggesting a limit of 3 percent would be appropriate. Deformation drift values are larger and vary significantly with frequency. Based on these data, rea- sonable limits for deformation drift would be 400 percent at 10 Hz, 300 percent at 1 Hz, and 200 percent at 0.10 Hz. Lim- its at other frequencies should be interpolated from these val- ues. Deformation uniformity should be limited to 20 percent, and phase uniformity to 3 percent. Based on an analysis of quality indices, the following lim- its should be used by dynamic modulus test users to identify potentially poor test data: • Load and deformation standard error: 10 percent • Load drift (absolute value): 3 percent • Deformation drift (absolute value): 400 percent at 10 Hz, 300 percent at 1 Hz, and percent % at 0.1 Hz • Deformation uniformity: 20 percent • Phase uniformity: 3 percent These limits are intended to help operators identify suspect data, so that such data can be evaluated and repeated if necessary. TABLE 27 Summary of quality indices for load and deformation

The limits should be set so that when tests are properly con- ducted by an experienced operator on a properly calibrated and maintained system, no more than about 5 percent of the test results should be identified as being suspect. It would not be efficient to identify a larger proportion of tests as being suspect, because this would result in unnecessary inves- tigations into test results and procedure and unnecessary repeated tests. 2.3.3.3 Flow Number The same approach described for the dynamic modulus was used to analyze the flow number data. Because there was only one test temperature and no differences in loading fre- quency, the flow number data were somewhat simpler. 2.3.3.3.1 General Trends. Figure 28 is a plot showing 95 percent confidence intervals for the coefficient of varia- tion (C.V.) in flow number for the various combinations of conditions (laboratory, device, mixture, confinement). The C.V. values range from about 12 to 66 percent with an aver- age of 31 percent. Most of the confidence intervals overlap, suggesting that there are not large differences in the standard 38 deviations relative to the mean values for most cases. The only pattern apparent from a visual examination of this plot is that the C.V. values for the 19-mm mixture appear to be gen- erally higher than those for the 9.5-mm mixture. Figure 29 is the corresponding figure for strain at flow, but in this case the confidence intervals are for standard deviation rather than coefficient of variation, because the range in this parameter was much smaller than for flow number and using C.V. did not significantly remove variability from the standard devia- tion values. Again, many of the confidence intervals overlap, suggesting that there are not many cases where large differ- ences exist in the variability in this measurement. One trend that does appear is that the standard deviations determined using confinement seem to be slightly larger than those deter- mined without confinement. Scatter plots with regression lines and confidence inter- vals were constructed to evaluate general trends between data generated in the two laboratories and by the two devices, both for flow number and strain at flow. For three of the four cases, the line of equality was captured by the confidence interval, indicating that there was not a strong indication of inequality. However, in the case of strain at flow, values generated by the Interlaken device tended to be higher at large strain values compared with those generated using the TABLE 28 Summary of drift and uniformity for load and deformation Figure 28. 95% confidence limits for flow number coefficient of variation at 45 °C.

Shedworks device, as shown in Figure 30. The deviation from equality is caused by the two measurements having the highest values; in both cases, these represent confined data for the 9.5-mm mixture. Flow in this mixture under confine- ment was particularly difficult to detect. The rate of change of permanent strain had a very long trough, making it dif- ficult to detect the minimum rate of strain using the speci- fied algorithm. The differences are, therefore, probably the result of differences in resolution of the measuring system caused by differences in electrical noise on the signal from the LVDT. 2.3.3.3.2 Detailed Analysis of Variability. In Table 29, a formal statistical comparison of standard deviations or coef- ficient of variation (C.V.) is presented, based on an F-test on the ratio of s2-values. In comparing variability for flow num- 39 ber, C.V. was used and treated as a normalized standard devi- ation, because otherwise the wide range in flow number val- ues could give misleading conclusions concerning variability. For flow number, the variability in the data generated at FHWA was somewhat greater than that produced at AAT, while the variability for data generated using the Shedworks device was somewhat greater than that measured using the Interlaken device. The variability in the data for the 19-mm mix was greater than that produced for the 9.5-mm mix, which is not surprising, though this was not observed in the modu- lus data. For the strain-at-flow data, the variability generated at the two laboratories was not significantly different, but the variability for data produced using the Interlaken device was higher than that produced using the Shedworks device. For strain at flow, the confined data also showed more variabil- ity than the unconfined data, which is not surprising because Figure 29. 95% confidence limits for strain at flow standard deviation at 45 °C. Figure 30. Comparison of strain at flow values generated using the ITC and IPC devices at 45 °C, including confined and unconfined data (R2 = 91.9%, adj. for d.f.).

using confinement provides additional complexity to the test and greater chance for error. 2.3.3.3.3 Detailed Analysis of Mean Response. Table 30 is a summary of statistical comparisons of mean responses for the different cases. For flow number, the mean responses for the two laboratories and two devices are not significantly different. The flow number for the 19-mm mixture tended to be significantly greater than that for the 9.5-mm mixture, and the flow number under confinement was larger than that mea- sured with no confinement. Both of these differences should be expected. For strain at flow, there again is no difference in mean response for the two laboratories. However, the Inter- laken device tended to show a somewhat larger value for strain at flow compared with the Shedworks device. The 9.5-mm mixture showed a larger value for strain at flow than the 19-mm mixture, while confinement tended to increase the strain at flow. The only discrepancy of concern is the slightly larger mean value for strain at flow measured using the Interlaken device compared with the Shedworks device. As observed for Figure 30 and the related discussion, this dif- ference is due to a greater response in only two cases—the confined tests for the 9.5-mm mixture, and flow in this mix- ture was particularly difficult to detect. 2.3.3.3.4 Analysis of Test Variability. A summary of the coefficient of variation (C.V.) values is given in Table 31. This table lists coefficient of variation values for different cases (laboratories, devices, mixtures, confinement), and over- 40 all coefficient of variations, both for flow number and strain at flow. Considering all flow number data, the overall C.V. was 34.6 percent, which is quite high compared with the C.V. for modulus of 13.0 percent. The C.V. for strain at failure was lower, with an overall average of 14.4 percent. The C.V. for the flow number from this study is somewhat higher than those reported for the large number of specimens testing during the Project 9-19 research. In the Project 9-19 research, coefficients of variation for the flow number test were reported to be 23.3 percent for 12.5 mm mixtures and 28.1 percent for 37.5 mm mixtures (17). These coefficients of variation are for one repli- cate measurement only. If the flow number test in practice is to represent the average of three determinations, then the coef- ficient of variation of the mean would be about 34.6/√3 = 20 percent, which is still high. Based on these tests, additional effort is needed to improve the precision of the flow number procedure before it can be used as a specification test. The statistical analysis of flow number test data resulted in several pertinent findings. These are summarized and dis- cussed below: 1. The flow number and strain at flow data were in gen- eral agreement among the devices and laboratories. 2. The variability in flow number data was slightly higher at the FHWA laboratory compared with the AAT lab- oratory and was higher for the Shedworks device than for the Interlaken device. 3. The 19-mm mixture exhibited greater variability in flow number data than the 9.5-mm mixture. 4. The variability in strain at flow was greater for the Inter- laken device compared with the Shedworks device and was also greater for confined conditions compared with unconfined conditions. 5. Most of the differences in mean response for both flow number and strain at flow were associated with differ- ent mixtures and/or different levels of confinement, which is to be expected. The one unexpected differ- ence in mean response was for strain at flow for the two measurements on the 9.5-mm mixture in confined testing, where the Interlaken device produced signifi- cantly higher values compared with the Shedworks device. 6. The overall variability of the flow number test data was much higher than that for the dynamic modulus, and the TABLE 29 Comparison of variability for flow number and strain at flow TABLE 30 Comparison of mean response for flow number and strain at flow TABLE 31 Coefficient of variation for flow number and strain at flow

data from this study showed higher variability than that reported in the original Project 9-19 research. The pri- mary difference in the flow number testing between this study and the Project 9-19 research was the algorithm used to calculate the flow number. In this study, the derivatives of the permanent strain curve were obtained using equally spaced sampling over the range of the data. In Project 9-19 logarithmic sampling was used in com- puting the derivatives, and these were further smoothed using a polynomial fit. This approach appears to fur- ther filter the data and provide less variable flow num- bers but significantly reduces the range over which the flow number can be detected. Using the Project 9-19 algorithm and 10,000 load cycles, the flow cannot be detected beyond about 8,000 cycles. 2.3.4 Functionality The primary objective of Project 9-29 was to stimulate the development of commercial equipment for performing the Project 9-19 simple performance tests in routine labo- ratory mixture design. For routine use, the functionality of the equipment is an extremely important consideration. The first-article specifications described minimum requirements for functionality, leaving ample opportunity for the manu- facturers to design user-friendly systems. In fact, perceived user-friendliness was a significant factor considered in the selection of the first-article manufacturers. Areas where the manufacturers could exercise substantial design freedom are listed below: 1. Aesthetics. Equipment size and shape, finish, noise lev- els, location of operator controls. 2. Safety Features. Emergency stops, safety interlocks, protection for load, pressure, and temperature. 3. Accessibility. Location of maintenance items and cal- ibration points. 4. Operation. Ease of operation, particularly specimen instrumentation, specimen insertion, and the helpful use of automation. 5. Controls. Logic and ease of use of the controls for tem- perature, load, and pressure. 6. Software. Function in addition to the minimum required by the specifications. Both first-article devices were demonstrated by the research team to the project panel, several engineers and technicians from a limited number of state highway agencies, and con- sultants. During the project, the Shedworks equipment was also demonstrated at various events by the FHWA in their Mobile Asphalt Laboratory. This provided the opportunity for some feedback from a wider range of individuals, includ- ing engineers and technicians from state highway agencies, hot-mix contractors, consulting firms, and universities. Over- 41 all, both first-article devices received favorable reviews by the research team, project panel, and most technicians and engineers who participated in equipment demonstrations. However, representatives from several hot-mix contractors expressed concern over the complexity of the equipment and its estimated cost. The sections that follow address the major strengths and weaknesses of the functional characteristics of the two first-article devices. 2.3.4.1 Interlaken Table 32 presents a summary of the assessment of the functional characteristics of the Interlaken Simple Perfor- mance Test Device. Although the equipment meets the min- imum requirements of the specifications, there are several areas needing improvement in future production units. The Interlaken first-article device looks very much like a prototype, primarily due to the configuration of the test cham- ber and the construction of the sight port. The finish of the metal work, particularly the horizontal surface around the test chamber and the chamber latch covers, also give the device a prototype appearance. Additionally, noise levels are high, and the overall size makes it difficult to use the equip- ment in a laboratory trailer. The safety features on the first-article device require some improvement. The overall control of the chamber is quite good. The chamber lift is interlocked with the chamber pres- sure so that the chamber cannot be raised while it is pressur- ized. On power loss, the chamber slowly lowers onto the latches. On loss of air pressure, the chamber slowly lowers to its seated position. Although well controlled, the mass of the chamber intimidates most users. Also the emergency stop is located in a position were it can be inadvertently activated during normal operations. Several critical operational areas require improvement. Perhaps the most important is the stability of the unique exten- someter system. In addition to the apparent errors discussed in the statistical analysis section, the extensometer system exhib- ited an unacceptable amount of initial drift during test ini- tiation and often had to be released and reapplied multiple times to obtain acceptable contact. This poor operational per- formance of the extensometer system negates the benefit afforded by an automated deformation measuring system. The extensometer system has the potential to simplify the dynamic modulus testing, but requires substantial improvement for use in production units. The configuration of the test chamber also presented operational difficulties. First, the specimen could not be seen through the sight port because there is no light inside the chamber. Although a lighted chamber would be an improvement, the engineers and technicians who performed the evaluation tests prefer a view of the specimen, instrumen- tation, and loading platens. The latches for the test chamber and the required position of the extensometer system result in a very confined space for inserting the specimen and platen assembly. This is particularly troublesome when confinement

is used and the hoses for the leak detection system need to be attached. Other more minor operational weaknesses in the device include the following: • Marginal cooling capacity. Although the system has sufficient capacity to return to the specified test temper- ature within the time stated in the specifications, it takes a long time initially to equilibrate the test chamber to temperatures below room temperature. • Poor leak detection system. The leak detection system was poorly assembled and required constant repair of joints and hoses. • Chamber O-ring seal. The O-ring seal at the bottom of the chamber is easily damaged and often sticks to the chamber when lifted. The equipment controls also require additional refinement and troubleshooting. Very high standard errors were observed for the unconfined 45 °C dynamic modulus data. Closer inves- tigation revealed that for these soft conditions, the hydraulic control system actually applies loading at 0.097 Hz. This dif- ference in loading rate results in very high standard errors. The actuator lacks a fine stroke control for initially seating the specimen prior to the start of testing. Occasionally a “run time error” is experienced during operation, which requires restarting the UniTest program. Overall, the UniTest software, as configured for the sim- ple performance tests, was found to be very logical and easy to use. Users found the summary dynamic modulus page, which shows test data and quality indicators for the entire frequency sweep extremely useful. The only weakness noted 42 in the software is that it is somewhat awkward to change between the three types of simple performance tests. 2.3.4.2 Shedworks Table 33 presents a summary of the assessment of the functional characteristics of the Shedworks Simple Perfor- mance Test Device. This equipment received high ratings for its appearance and many operational characteristics, but still requires some improvements for future production units. The Shedworks device does not look like a first-article. Operators and observers were impressed with the quality of the metal work, the quality of the machine work, and the over- all finish of the device. They also commented very positively on its compact size and relatively quiet operation. The abil- ity to move the hydraulic power supply to a remote location is another strength of the Shedworks design. The device received mixed reviews for its safety features. It includes a hands clear safety feature that requires the oper- ator to hold two buttons to close the test chamber. Other safety features, however, require improvement. The chamber lift is not interlocked with pressure, allowing the operator to open the chamber while it is pressurized. On loss of power, the chamber closes too rapidly, and the emergency stop is located in a position where it can be inadvertently activated during testing. The Shedworks device was found to be very user-friendly. The automated gauge point system is well designed and worked extremely well. This system combined with the clip- on LVDTs produced a very rugged, practical specimen defor- mation measuring system. The chamber provided a view of TABLE 32 Rating of functional characteristics of the Interlaken simple performance test device

the specimen, instrumentation, and loading platens during testing, which operators and observers found to be essential. Finally, the temperature control system has sufficient excess capacity to allow for very rapid heating and cooling of the chamber. The Shedworks first-article device requires improvements in machine control and some software refinements for pro- duction units. The most urgent improvement is to remedy the situation where the control software locks up during the flow number testing when the maximum number of load cycles is reached. When this situation occurs, the software must be restarted. Instability of the hydraulics was observed when the system was cold. This instability was characterized by a rapid, uncontrolled oscillation of the loading actuator and was only observed on start-up when the hydraulic oil was cold. 43 The software used in the Shedworks first-article device also requires further refinement. Although the software was found to be user-friendly and relatively easy to learn, the optional IPC data analyses must be removed from the soft- ware for production units to eliminate confusion caused by multiple data analysis methods. Also, the software displays too many significant digits, giving the impression that there is a large amount of noise in the transducer signals and mak- ing it difficult to assess the status of the transducers quickly. Finally, the layering of the windows in the software some- times covers important control information. For example, the LVDT levels window is sometimes not visible during test- ing. Having this window as a bar that is constantly displayed would allow operators to quickly view the status of the trans- ducers at any time. TABLE 33 Rating of functional characteristics of the Shedworks simple performance test device