Highway Infrastructure Inspection Practices for the Digital Age (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

9   Literature Review Introduction This chapter documents the information collected from the literature review of technologies for highway infrastructure inspection during construction and asset management. The objective of this chapter is to set the background and context for the findings from the survey and case examples presented in Chapters 3 and 4, respectively. This chapter discusses a range of technolo- gies that are used in highway infrastructure inspection and focuses on the following technology areas: (1) geospatial technologies, (2) remote sensing and monitoring technologies, (3) mobile devices and software applications, and (4) nondestructive evaluation methods. This chapter also discusses typical technologies within each of these four technology areas. It is noted that there is a blurred boundary among the four areas. As a result, most technologies can be classified in more than one area. For example, some technologies in the geospatial technology area are also used as remote sensing technologies or within mobile devices. Similarly, nondestructive evalu- ation methods are often developed and used along with remote sensing devices. Geospatial Technologies Geospatial technologies have rapidly evolved over the past decade. In the construction industry, geospatial technology is a foundational element of connecting the design and construction of a highway project (FHWA 2018). Many decisions in highway construction projects are made on the basis of the data representing the construction progress and existing conditions of high- way assets. Geospatial technologies have proved to be an effective tool for collecting accurate geospatial data in near real time (Mallela et al. 2018). There is a wide range of geospatial tech- nologies, and their applications provide numerous benefits to highway construction projects and programs. This section discusses the use of the following technologies for highway infrastructure inspection: • Global Navigation Satellite Systems (GNSS)/Global Positioning System (GPS) • Geographic Information Systems (GIS) • Unmanned aircraft systems (UASs) • Robotic total stations (RTSs) • Terrestrial photogrammetry (TP) • E-ticketing Global Navigation Satellite Systems and the Global Positioning System GNSS is the standard term for satellite navigation systems (i.e., satellite constellations) that provide autonomous geospatial positioning anywhere on Earth. The main GNSS constellations C H A P T E R   2

10 Highway Infrastructure Inspection Practices for the Digital Age include GPS, GLONASS, Galileo, and BeiDou. These satellites provide signals from space and transmit positioning and timing data to GNSS receivers. The use of multiple satellites reduces delays in finding adequate ranges (Mallela et al. 2018) and ensures that location data are accu- rate, redundant, and available at all times. If one satellite does not provide a quality position or fails to operate, GNSS receivers can pick up signals from other systems. Figure 2.1 shows a GNSS-guided motor grader. GPS has been fully operational for more than two decades and has become a vital tool in the construction industry (Mallela et al. 2018; Ogaja 2011). GPS consists of up to 32 medium Earth orbit satellites in six different orbital planes. The exact number of satellites in GPS varies because older satellites are retired and replaced. A number of GPS-based systems have been proposed or implemented to facilitate highway infrastructure inspection through the identification and tracking of materials on construction sites. Typically, GPS and GNSS technologies include three main segments as follows: • Space segment—This segment consists of satellites that continuously broadcast position and time data to GNSS receivers. • Control segment—This segment consists of ground stations that monitor, track, and collect the satellite broadcast signals. • User segment—This segment consists of receivers, processors, and antennas that allow opera- tors to determine the position, velocity, and time of the operator’s location. Table 2.1 summarizes the typical use of GNSS/GPS technologies as reported by state DOTs, construction contractors, and instrument developers and service providers through a series of interviews (Mallela et al. 2018). Table 2.1 shows that both state DOTs and construction con- tractors use GNSS/GPS technologies for highway inspection during construction and for asset management. Geographic Information Systems GIS is a computer-based system that provides a set of tools for the creation, management, analysis, and display of spatial data. Deploying GIS for transportation (GIS-T) was initiated in the 1980s. The application of GIS-T in its early days was for the 2D cartographic display of roadway data. Since then, GIS has evolved along with the ways it is used by federal, state, and local agencies for infrastructure planning and management, routing and scheduling, and other purposes (Butler et al. 2019). GIS was initially deployed as a means of demonstrating the content of text-based inventory database systems. With its substantial deployment in many areas of the transportation industry, GIS is increasingly becoming the common roadway inventory database Figure 2.1. Use of GNSS for subgrade (Source: Mallela et al. 2018).

Literature Review 11   (Butler et al. 2019). As a result, GIS serves as a data management tool integrated into civil infra- structure management, which is an “asset management system that starts with planning the construction or modification of a transportation asset, moves through design and construc- tion, and, once built, transitions to [a] deployment, operations, maintenance, and performance evaluation system” (Butler et al. 2019). Figure 2.2 shows the data flow between GIS and building information modeling (BIM) platforms as a means of providing roadway inventory updates when a construction project ends. Integrating GIS and BIM offers opportunities to streamline agencies’ business processes through visualizing, sharing, analyzing, and monitoring asset data. Applications Use of GNSS/GPS for highway infrastructure By State DOTs By Construction Companies By Instrument Developers/ Service Providers Topographic surveying Earthwork Paving Roadway design AMG and control Verification As-built surveys Site/progress monitoring Inspection Quality assurance/quality control Asset management Source: Adapted from Mallela et al. 2018. Table 2.1. Typical use of GNSS/GPS for highway infrastructure. Figure 2.2. GIS and BIM data flow (Source: Butler et al. 2019). GIS enhances the field of transportation asset management. Maps help leaders see the extent of problems, understand the geographic effects of their decisions, and ultimately make more informed decisions. Maps can also help the public see and understand the far-reaching importance of the transportation assets they use every day. GIS enables transportation agencies to show information about their assets on maps that both technical and non-technical audiences can understand (Hector-Hsu et al. 2012).

12 Highway Infrastructure Inspection Practices for the Digital Age Unmanned Aircraft Systems (UASs) There has been increasing attention and resources dedicated to the application and opera- tion of UASs over the past decade. In 2016, the Federal Aviation Administration (FAA) pub- lished the final rule for civil operations of UASs. In 2017, the U.S. DOT and FAA established the UAS Integration Pilot Program to (1) accelerate the safe integration of UASs into the national airspace system, (2) address ongoing concerns regarding potential security and safety risks, (3) promote innovation, and (4) identify the most effective models of UAS integration (Banks et al. 2018). UASs have been investigated for a number of highway infrastructure applications. Pistorius (2017) and Pecoraro et al. (2017) summarize some uses for UASs in construction: • Pre-project assessments and project survey data—UASs can offer aerial images of a project site much more accurately, realistically, and timely than traditional aircraft photographs are able to. • Conducting aircraft surveys and site mapping—UASs can provide real-time aerial views of key project areas, allowing management and construction staff to monitor operations and performance, which in turn supports decision making. • Site inspections and surveillance—UASs can be used for inspection of bridges, high-tension electric wires, remote sites, and other areas that are difficult to access. Unmanned aerial vehicles (UAVs) equipped with camera and video equipment provide site security. • Asset tracking and management—UASs can be used to keep track of the movements of machines, equipment, tools, vehicles, and people. UASs can monitor workers on site and are convenient tools to keep track of how many employees are working in sensitive or hazardous areas. • Monitoring the movement of materials, stockpile reporting, and inventory management—UASs are used to keep a record of material being delivered, stored, and installed. • Enhanced safety—UASs can provide real-time data of safety violations or situations that might be unsafe during the construction process. • Enhanced 3D modeling—UAS data are acquired in real time and integrated with mapping and BIM models. Many state DOTs have used UASs for different purposes, such as site surveying, tracking construction progress, monitoring roadside environmental conditions, or traffic management and safety improvement (Pecoraro et al. 2017). Banks et al. (2018) summarizes the typical uses of UASs for highway infrastructure inspection as follows: • Visual inspection • Jobsite volumetric documentation • Work zone safety • Surveying and mapping • Environmental compliance • Bridge inspection • Photography • Confined spaces assessment Additionally, researchers have applied UASs in a number of ways, including monitoring traffic; inspecting structures; assisting with construction safety inspections; inventorying roadside conditions; surveying and mapping topographic features; monitoring construction progress; estimating earthwork volumes; identifying potential avalanches near roadways; monitoring unstable slopes and mapping landslides; and reconstructing and documenting crash scenes (Mallela et al. 2018). Table 2.2 summarizes typical applications of UASs associated with benefits and limitations related to highway infrastructure inspections.

Literature Review 13   Robotic Total Stations Robotic total stations (RTSs) allow for more technological convenience by offering remote- control abilities to perform more calculations and inspections in less time and with less staff than a traditional total station. RTSs allow for increased safety in challenging terrain because of their unique ability to operate at a distance. RTSs are a well-established technology and one of the most common geospatial tools used by state DOTs for construction inspection (Mallela et al. 2018). Total stations typically include the following capabilities: • Prism measurement—The ability to measure angles and distances between points and coordinates; • Reflectorless measurement—The ability to collect features not accessible with a prism (e.g., obtaining measurements on the sides of bridges or steep slopes); • Imaging—The ability to collect 360-degree digital images; • Integrated GNSS—The ability to record the instrument’s location, providing added flexibility in georeferencing total station data; and • Integrated LiDAR scanning—Using built-in multi-function systems to obtain small point clouds directly connected to the total station (Mallela et al. 2018). Figure 2.3 shows an example of a GT-1200 RTS that is designed to better handle data and has built-in, field-to-office connectivity. Mallela et al. (2018) report the following typical applications of RTS for highway infrastruc- ture inspection as used by state DOTs: • Earthwork • Paving • Verification • As-built surveys • Site/progress monitoring Source: Adapted from Mallela et al. 2018. Areas of Applications Studies/Authors Benefits Limitations Construction safety inspection Gheisari et al. 2014 Increased efficiency; permanent digital records Potential distraction and collision hazards Roadside condition inventorying, assessment, and inspection Barfuss et al. 2012; Hart and Gharaibeh 2011; Zhang 2008 Permanent digital records; “bird’s-eye view” not possible with vehicle- mounted imaging systems Data collection of visible assets via “bird’s-eye view” only Topographic mapping and estimating earthwork volumes Brooks et al. 2014; Hugenholtz et al. 2015; Judson 2013; Siebert and Teizer 2014 Quick data collection to produce preliminary 3D mapping products Less accurate than other technologies Monitoring construction progress and status Lin et al. 2015; Wang et al. 2014; Zollmann et al. 2014 Quick data collection to produce 3D as-built models; permanent digital records Lack of automation tools to create building information model components Monitoring unstable slopes Lucieer et al. 2014; Niethammer et al. 2010 Inexpensive and quick data collection over uneven terrain Requires ground control points or aerial targets Table 2.2. Typical application of UASs for highway infrastructure inspection.

14 Highway Infrastructure Inspection Practices for the Digital Age • Inspection • Quality assurance/quality control • Asset management Photogrammetry Technology Photogrammetry is the process of extracting geometric measurements from photographs. A wide range of studies discuss the application of photogrammetry in highway construction. The two main types of photogrammetry are aerial photogrammetry and terrestrial photogram- metry. While conventional aerial photogrammetry is suitable for acquiring data over large areas, it often involves the expense of mobilizing an aircraft to the project site. Because UASs are rela- tively inexpensive and are able to perform data collection, using UASs with photogrammetry can be a viable tool to reduce the costs of conventional aerial photogrammetry (Mallela et al. 2018). Terrestrial photogrammetry requires less investment and is suitable for mapping movement- related issues such as monitoring highway construction progress. While photogrammetry is a mature discipline, structure from motion (SfM) is a relatively new application in the construc- tion industry. SfM is a photogrammetric approach developed on the basis of the advanced image matching algorithms that originated from the computer vision (Mallela et al. 2018). SfM typi- cally involves high-resolution, close-range images. As a result, SfM is an effective tool to extract detailed 3D spatial information of individual objects on a construction site (Mallela et al. 2018). On the basis of a series of interviews, Mallela et al. (2018) have shown that the typical applica- tions of terrestrial photogrammetry and SfM are as follows: • Earthwork • Quality assurance/quality control activities • Inspection • Asset management • Verification • As-built surveys Additionally, researchers have applied terrestrial photogrammetry and SfM to highway infra- structure inspection in several ways, including as-built data, real-time 3D construction field data, asset condition evaluation, bridge inspection, construction project progress tracking, and Figure 2.3. GT-1200 robotic total station (Source: Topcon 2021).

Literature Review 15   construction quality control (Mallela et al. 2018). Table 2.3 summarizes typical applications of terrestrial photogrammetry and SfM related to highway infrastructure inspections. E-Ticketing E-ticketing technology allows users to collect and document load delivery data electroni- cally, making construction inspections and management safer and more efficient. FHWA promoted e-ticketing as an e-construction technology that can be collaborative and mutually beneficial to both state DOTs and their contractors. Several state DOTs (e.g., Florida, Ohio, Alabama, Iowa, Minnesota) have experience using e-ticketing for their highway construc- tion projects and indicated that the typical benefits of e-ticketing implementation include improved safety, enhanced efficiency, and an offset to staff reductions (FHWA 2021a). E-ticketing has gained substantial momentum over the past 5 years. Additionally, because of the COVID-19 pandemic, many state DOTs are increasingly using or implementing e-ticketing technolo- gies to reduce the risk of virus transmission and improve the safety of inspectors and on-site personnel. The current e-ticketing practices typically involve a third-party provider connecting to the loadout system of a material producer (most often asphalt or concrete) to collect material and load information. In some cases, GPS units are installed on mobile equipment such as haulers and pavers and are accessed through a web-based interface to allow for real-time tracking of deliveries and reporting of electronic load information (Dadi et al. 2020). The three main types of e-ticketing projects are hot mix asphalt (HMA), Portland cement concrete (PCC), and aggre- gate. Other types of projects that are less common e-tickets include structural steel, rebar, guard- rail, signs, and millings. Table 2.4 summarizes the experiences of early adopters of e-ticketing and effective practices of their pilot efforts. Source: Adapted from Mallela et al. 2018. Areas of Applications Studies/Authors Remarks Visualization and progress monitoring of construction projects Golparvar-Fard et al. 2009b 4D models generated from SfM and close-range photogrammetry As-built data, quantity surveying in construction management, real- time 3D construction field data Dai and Lu 2010 Close-range photogrammetry using off-the-shelf, portable digital cameras Monitoring of structures for damage during construction Luhmann and Tecklenburg 2001 Close-range photogrammetry for monitoring buildings for damage during construction Asset condition evaluation Olsen et al. 2013b A synthesis of photogrammetry with video logging, GPS, and GIS for assessment of highway conditions Bridge inspection and historic bridge documentation Jáuregui et al. 2006 Close-range photogrammetry for bridge inspection and historic bridge documentation Construction project progress tracking El-Omari and Moselhi 2008 Integration of LiDAR, RFID/barcode, and close-range photogrammetry for progress measurement on construction sites. Close-range photogrammetry and Automated Construction Project Monitoring system Quality control Dai and Lu 2012 SfM using site photos for 3D modeling and quality control Table 2.3. Typical application of terrestrial photogrammetry and SfM for highway infrastructure inspection.

16 Highway Infrastructure Inspection Practices for the Digital Age Remote Sensing and Monitoring Technologies Remote sensing and monitoring technologies, as defined in this study, involve the use of sensors and devices to provide inspection or monitoring at a physical distance from the point of assessment. The term “remote sensing” is used to describe 3D data remotely acquired using technologies such as LiDAR and other 3D imaging devices. There is a wide range of sensors (e.g., accelerometers, remote cameras, or traffic loops) that are configured to operate remotely and provide data access via radio, cellular networks, the internet, or other communication means (Maier et al. 2018). The following sections discuss the typical remote sensing and monitoring technologies used in highway infrastructure inspection, including LiDAR/3D laser scanning, radio-frequency identification (RFID), intelligent compaction, and barcodes and readers. Light imaging, Detection, and Ranging (LiDAR) LiDAR, which is also considered to be a geospatial technology, is an optical remote sensing technology typically used for measuring the distance between a surface and the sensing units. LiDAR is effectively used to acquire X, Y, Z (i.e., 3D) positions of any surface within visual sight of the sensing unit. There are three main LiDAR applications: aerial LiDAR, mobile LiDAR (e.g., a system that can be attached to a vehicle such as a truck or unmanned aircraft), and static LiDAR (e.g., a system mounted at a single location) (Maier et al. 2018). Table 2.5 shows typical project charac- teristics of the different LiDAR methods. E-ticketing provides all stakeholders with an electronic means to produce, transmit, and share materials data and to track and verify material deliveries with enhanced safety, streamlined inspections, and improved contract administration processing. Using electronic ticket exchanges enables access via mobile devices and simplifies handling and integration of materials data into construction management systems for acceptance, payment, and source documentation (FHWA 2021a). Material Type Procurement DOTs Implemented Effective Practices Asphalt • Bid item • Change order • Developed own system • Purchased by DOTs AL, FL, IA, KY, MN, MO, ND, PA, UT, VA • Early stakeholder communication • Hands-on training • Data storage and transfer plan Concrete Change order FL, IA • Contractor-producer early buy- in • Contractor-producer communication • Thorough stakeholder training Aggregate Change order VA Millings Bid item PA • Early stakeholder communication • Hands-on training • Data storage and transfer plan Source: Adapted from Dadi et al. 2020. Table 2.4. Summary of e-ticketing experiences and effective practices.

Literature Review 17   Static LiDAR collects highly accurate data, but it is comparatively much slower in data collec- tion than mobile and aerial LiDAR and it exposes DOT workers to more traffic and hazard risks. Both mobile and aerial LiDAR provide mapping-grade accuracy at high rates of travel. Mobile LiDAR applications involve digital highway measurement vehicles, LiDAR, inertial navigation systems, and GPS to provide measurements of elements such as pavement markings, pave- ment cross sections, shoulders, and curbs (Ogle 2007; Olsen et al. 2013a). Aerial LiDAR systems can collect data when traveling at 115  miles per hour at an elevation of about 1,640 ft (Dye Management 2014). It is noted that aerial LiDAR systems include airborne, helicopter, and UAS platforms. Mobile LiDAR systems include handheld and vehicle plat- forms. Table 2.6 summarizes the description, capabilities, and limitations of each of these types of LiDAR systems. It is noted that the recent development of aerial LiDAR involved improving data processing by using 3D point clouds. The cloud-based data processing approach has resulted in enhanced flexibility and faster turnaround times between the field and the office. Many state DOTs have used LiDAR technology for highway infrastructure inspection–related applications. Table 2.7 summarizes the typical uses of LiDAR systems for highway construction and inspection as reported by state DOTs, construction contractors, and instrument developers and service providers (Mallela et al. 2018). One of the key benefits of LiDAR technology is that its acquired data are useful for a number of applications. The data collected when using LiDAR can also be mined for additional informa- tion that can serve as suitable input for different applications. NCHRP Report 748: Guidelines for the Use of Mobile LIDAR in Transportation Applications (Olsen et al. 2013a), shows various uses of mobile LiDAR related to construction delivery, including the following: • As-built and maintenance documentation—The data are integrated into a centralized database that is continuously updated for future planning, maintenance, and construction. • Pavement smoothness and quality determination—Data collected at higher resolutions can be used to evaluate pavement smoothness and quality. • Construction automation and quality control—Change detection and deviation analysis soft- ware uses design models to identify deviations from LiDAR point clouds for construction quality control. • Performing quantity takeoff—LiDAR data are used to determine lengths, areas, or volumes of construction quantity. Aerial LiDAR Mobile LiDAR Static LiDAR • Mainline lengths > 1,300 ft • Large areas and wide corridors • Large bridge replacements • Variable terrain • Rural reconstructions • Areas with limited foliage • Long, rural corridors • High-speed corridors • Corridors with high volumes • Multilevel interchanges • Resurfacing projects with cross-slope or super- elevation corrections • Data collection time constraints • Mainline lengths < 1,300 ft • Small areas • At-grade intersections • Low-volume and low- speed roadways • Flat terrain • Small bridge replacements • Urban resurfacing projects with drainage or cross-slope repairs • Interstate widening Source: Adapted from Maier et al. 2018. Table 2.5. Project characteristics of the different LiDAR methods.

18 Highway Infrastructure Inspection Practices for the Digital Age LiDAR Type Description Capabilities Limitations Aerial-Airplane (airborne) Sensor attached to fixed- wing aircraft at 1,000 m or more above ground; co- acquired photographic images are becoming more common • Rapid coverage over large areas • Fairly uniform sampling • Can collect other remote sensing data simultaneously • Large footprint • Poor coverage on vertical faces • Flight logistics Aerial-Helicopter Sensor mounted to a helicopter flying closer to the ground • Similar to airborne, but closer to ground • Flight logistics may be complicated Aerial-UAS Lightweight sensor mounted to an unmanned aerial system; flight heights are typically less than 150 m • Detailed information for a site • Pre-programmed flight paths • Nadir and oblique scanning possible • Short flying time limits to relatively small areas • Few systems available, experimental Mobile-Handheld Sensor carried in hand or on a backpack frame • Flexible system • Indoor/outdoor • Only one person required • Slower than most other methods for large areas Mobile-Vehicle Sensor mounted to a vehicle and data are collected kinematically while a vehicle is in motion • Fast coverage along highways • Limited to navigable paths • Obstructions from traffic Static Instrument is mounted to a tripod. Photographic images are often co- acquired; typically implemented only for smaller sites • Highest resolution • Highest accuracy • Some flexibility • Indoor/outdoor • Slower than other techniques • Non-uniform sampling Source: Adapted from Mallela et al. 2018. Table 2.6. Summary of different LiDAR systems. Applications Use of LiDAR By State DOTs By Construction Companies By Instrument Developers/ Service Providers Topographic surveying (1) (2) (3) (1) (2) (3) (1) (2) (3) Earthwork (1) (3) (1) (2) (3) (1) (2) (3) Paving (1) (2) (1) (1) (2) (3) Roadway design (1) (2) (3) (3) (2) (3) AMG and control (1) (2) (3) Verification (1) (2) (1) (1) (3) As-built surveys (1) (2) (1) (2) (3) (1) (2) Site/progress monitoring (1) (2) Inspection (1) (2) (3) (2) (1) Quality assurance/quality control (1) (2) (1) (2) (3) (1) (3) Asset management (1) (2) (3) (1) (2) (3) (2) Source: Adapted from Mallela et al. 2018. Notes: (1) denotes static LiDAR, (2) denotes mobile LiDAR, and (3) denotes aerial LiDAR. Table 2.7. Typical uses and methods (static, mobile, aerial) of LiDAR for highway construction and inspection.

Literature Review 19   • Virtual and 3D design—LiDAR data can be used for clash detection by checking for intersec- tions of proposed objects with existing objects modeled in the point cloud. • Inspections—LiDAR can provide overall geometric information and an overall condition assessment. Radio-Frequency Identification Research on RFID technologies has grown exponentially over the past two decades. Jaselskis et al. (1994), one of the seminal studies on RFID, indicated three main applications of RFID in the construction industry: (1) monitoring concrete deliveries, (2) tracking workers and equip- ment, and (3) managing critical materials. Among these three applications, monitoring concrete deliveries is a common highway infrastructure inspection activity. Figure 2.4 shows an over- view of an RFID-based concrete tracking system. In essence, RFID tags are pinned to trucks or materials, the tags are encoded with information, and then readers extract and transmit data to a central database. Schwartz (2015) highlighted the following specific RFID applications in highway construction: • Tracking Portland cement concrete (PCC) test specimens, • Maturity monitoring in-place PCC, • Tracking construction materials at project sites (e.g., fabricated pipe), • Construction tool inventory/usage control/monitoring, and • Identifying buried infrastructure locations (e.g., cables and pipes). Schwartz (2015) also identified the typical aspects of RFID technology, including the following: • Physical size of the tag, • Operating frequency (low, high, ultrahigh, and microwave; this influences tag size and read range), • Type of tag (active or passive; this influences cost, power consumption, and maximum read range), • Read range (including high-gain antenna design), • Durability/survivability (e.g., strength of tag encapsulation; maximum/operating temperature ranges), and • Cost of the expendable tag. Li and Becerik-Gerber (2011) showed that when RFID is combined with other technologies such as GPS or GNSS, it can streamline work at construction sites by eliminating the burden of manually tracking individual components, assisting with tool management, tracking the status of component assembly, and facilitating safety, quality assurance, and quality control. For example, Castro-Martínez et al. (2019) developed an RFID-based method to identify cracks in connections on steel girder bridges and other metallic structures. Using a passive RFID tag, damage is assessed on the basis of variations in backscatter power. Several state DOTs have used RFID technologies for their highway construction projects. For example, the North Carolina DOT has implemented an RFID-based tracking system for precast and pour-in-place elements. The RFID system was introduced in response to problems that arise from merely stamping elements with inventory numbers: poor tracking, misapplied stamps, incorrect paperwork, and uninspected materials arriving on project sites (Peoples 2018). The Georgia DOT has experimented with the use of read-only RFID tags to track construction materials that undergo lab testing. The agency tracked material arrival at the lab facility, assign- ment for testing, delivery to lab for testing, completion of testing, and the logging of results, with all of the information synced automatically with a database. The Maryland State Highway Administration studied the feasibility of using RFID tags and barcodes to track material field

20 Highway Infrastructure Inspection Practices for the Digital Age Figure 2.4. RFID-based concrete tracking system (Source: Jaselskis et al. 1994).

Literature Review 21   samples in laboratories (U.S. DOT 2018). The Alaska DOT and FHWA studied the use of RFID to trace pay items (e.g., asphalt, base course, borrow, and riprap). Basically, this amounted to an e-ticketing pilot study, with RFID tags placed on dump trucks to log when they left and entered project sites (U.S. DOT 2018). Intelligent Compaction Intelligent compaction (IC) is an equipment-based technology used to improve the quality, uniformity, and long-lasting performance of pavements. IC technology can be applied to all pavement layer materials and soils aggregates. The typical parts of IC machines include vibratory rollers with accelerometers mounted on the axle of the drums, a GPS device, infrared tempera- ture sensors, and onboard computers that can display color-coded maps in real time to track roller passes, surface temperatures, and stiffness of compacted materials (Torres et al. 2018). Figure 2.5 shows an example of IC for asphalt pavement. IC has been used by a number of state DOTs. For example, 13 state DOTs participated in an FHWA-sponsored Transportation Pooled Fund (Chang et al. 2011) study entitled Accelerated Implementation of Intelligent Compaction Technology for Embankment Subgrade Soils, Aggre- gate Base, and Asphalt Pavement Materials. The primary purpose of the study was to demonstrate and evaluate IC technologies through multiple field projects. FHWA (2013a) highlighted several benefits of using IC for highway construction. The main benefits related to highway infrastructure inspection include the following: • Reduced material variability—Using IC equipment allows contractors to closely monitor the stiffness of the material, which leads to less variability in the end result. Lower variability results in better pavement performance and reduced maintenance and repair costs. • Identification of areas that are not compactible—Using IC equipment allows contractors to identify potential areas (e.g., areas that fail to reach the target compaction level) for reworking the defective material or removing and replacing it. • Ability to make midcourse corrections—IC equipment allows contractors to correct compac- tion problems in a subsurface layer before additional layers are placed, which ensures that subsurface problems do not affect the entire road surface. • Ability to maintain construction records—Using IC equipment allows users to download data from IC operation, along with GPS coordinates of compaction activity, into construction- quality databases. These databases can be stored electronically for future reference. Figure 2.5. An example of IC for asphalt pavement (Source: Torres et al. 2018).

22 Highway Infrastructure Inspection Practices for the Digital Age Barcodes Barcodes have been successfully implemented in the construction industry to improve the accessibility of information, as scanning barcode data allows for faster, more accurate, and more secure transfers of data compared with manually entered data (Lee et al. 2018). Barcodes can be formed in three different formats: one-dimensional (1D), two-dimensional (2D), or three-dimensional (3D). The 1D barcode is linear and consists of parallel lines and spaces that are vertically oriented. One-dimensional barcodes are simple to create and easy to read. The 2D barcode has an overall square shape, inside of which is a unique arrangement of rectangles. A common example of a 2D barcode is the now-ubiquitous Quick Response (QR) code, which can be scanned with a mobile device or camera. QR codes are used in construction mainly for version checking of design documents. QR codes are effective in reducing the amount of rework by providing project personnel with access to the latest version of the drawings. Lee et al. (2018) found that using QR codes can provide project teams with rapid and reliable access to informa- tion and documents required in field operations. Finally, 3D barcodes resemble 2D barcodes; however, the internal rectangles extend to varying heights, which are measured and interpreted by scanners (Nikolow 2012). Figure 2.6 illustrates a process of using barcodes for automated material deliveries for a con- struction project at the Indiana DOT. The process includes five steps: (1) generating a ticket using barcodes, (2) delivering ticket barcodes to the jobsite, (3) Indiana DOT inspectors scanning the Figure 2.6. Automated materials delivery using barcodes (INDOT = Indiana DOT) (Source: Lee and McCullouch 2008).

Literature Review 23   barcodes, (4) transmitting the scanned data to the contractor as a receipt, and (5) transmitting the scanned data to the material supplier as a record. State DOTs have experimented with barcodes and have typically used barcodes with other technologies. An Iowa DOT pilot project used QR codes affixed to the dashboards of haul trucks to monitor concrete deliveries (Shepard 2017). The Louisiana Department of Transportation and Development used a barcode-based automated material tracking system to eliminate manual data entry by scanning asphalt delivery tickets when trucks entered project sites (Icenogle et al. 2013). In Georgia, Tsai and Wang (2016) briefly discussed a plan to use barcodes and GPS devices to inventory newly installed highway signs. As part of its e-construction, the North Carolina DOT envisions full implementation of barcodes and RFID tags for material product approvals (FHWA 2019a). Mobile Devices and Software Applications Construction inspectors, on average, spend half of their shifts collecting inspection infor- mation in the field and the other half performing administrative tasks such as searching for relevant documents (e.g., plans or specifications) or entering field-collected information into a computer-based system (Snow et al. 2013; Yamaura and Muench 2018). Mobile devices and soft- ware applications provide state DOTs with the opportunity to improve the process of collecting, documenting, and distributing project inspection information. A wide range of mobile devices and software applications are used in the field for highway inspections during construction and maintenance of assets. The following sections discuss 3D engineered models, automated machine guidance (AMG), handheld data collectors, and virtual reality (VR) and augmented reality (AR). 3D Engineered Models 3D engineered modeling is a mature technology that enables faster, more accurate, and more efficient planning and construction. FHWA has supported the use of 3D engineered models for highway construction inspection through its EDC program (FHWA 2017a). FHWA (2017a) found that there is an increasing number of construction projects in which the inspection team is using 3D models for real-time verification and collecting post-construction survey data for measuring payment quantities. Figure 2.7 illustrates the use of 3D engineered models for construc- tion inspection processes across three main phases: pre-construction, during construction, and post-construction. For the pre-construction phase, inspectors use 3D models to develop a plan for inspection. The plan helps inspectors (1) assess the impacts of differences between survey data and field con- ditions, (2) manage the digital data, (3) define the tools and methods for inspection, (4) identify Figure 2.7. Use of 3D engineered models for construction inspection (Source: FHWA 2017a).

24 Highway Infrastructure Inspection Practices for the Digital Age sources of inspection support during construction, (5) establish the process for measuring and calculating quantities for payment, and (6) determine the needs and formats for 3D as-built records (FHWA 2017a). During the construction phase, 3D models provide inspectors with an independent and trans- parent construction inspection process to evaluate field conditions relative to the drawings and specifications. 3D models also help the inspectors effectively and properly document and deter- mine pay quantities (FHWA 2017a). It is noted that 3D models provide many ways to view the data collected, including visual field representations and spreadsheets. For the post-construction phase, 3D models help the inspector understand the accurate as-built conditions of the project. 3D engineered models also provide reliable data for long-term documentation and asset management. The Iowa DOT indicated that 3D engineered models can be updated throughout construction using a mobile computer system (Reeder and Nelson 2015). The 3D model becomes the record drawing, which is critical for highway infrastructure inspection for maintenance of assets. The 3D engineered model serves as a base map for future maintenance and updating of asset management systems (Reeder and Nelson 2015). The EDC-3 Final Report (FHWA 2017a) found that the typical challenges for inspectors when using 3D engineered models in construction inspection are: • Needing education and training for manipulating 3D models; • Having difficulties obtaining a reliable set of 3D data; • Needing equipment, software, and tools required to use 3D models in the office and in the field; • Lacking understanding of how the 3D data reflects the contract plans, specifications, and field conditions; and • Needing new skill sets to perform field calibration of the model and survey methods and tools. Automated Machine Guidance Automated machine guidance (AMG) uses a combination of 3D modeling data and GPS technology to guide construction equipment and enhance construction efficiencies. The EDC-3 Final Report (FHWA 2017a) highlighted that “a critical success factor for AMG is simultane- ously using survey technology for real-time quality control (QC) feedback.” Figure 2.8 shows a motor grader equipped with AMG technology. Figure 2.8. A motor grader equipped with AMG (Source: Reeder and Nelson 2015).

Literature Review 25   The typical projects that are good candidates for AMG are those that involve large amounts of earthwork/paving or new alignments, or projects that require accurate digital terrain models (FHWA 2013b). The reported benefits of using AMG for highway infrastructure inspection include: • Higher levels of accuracy and precision of construction work compared with traditional con- struction methods, • Better control of elevation and cross slope for asphalt paving projects, • Reduced errors and required rework, • More accurate calculations when performing quality assurance and quantity calculations, • Better calibration and control of paving equipment with fewer errors, • Reduction of survey stakes and the costs savings associated with less field survey requirements, • The operator receives real-time feedback on the behavior of construction equipment in rela- tion to 3D data, and • Increased potential for electronic “as-builts” (FHWA 2013b). It is noted that some state DOTs have created special provisions that allow contractors using AMG to provide equipment and training for inspectors. Maier et al. (2018) noted that there is an evolution in special provisions to allow the use of AMG and real-time verification. For example, the Florida DOT, Iowa DOT, Kentucky Transportation Cabinet, New York State DOT, Utah DOT, and Wisconsin DOT have special provisions, developmental specifications, and standard specifications for the use of AMG (Maier et al. 2018). Table 2.8 shows the applicable AMG methods associated with typical project types. Handheld Devices Handheld devices are defined as “self-contained electronic devices that fit in the palm of a user’s hand and possess, at a minimum, enough computer processing power to surpass the func- tions of an electronic personal organizer and to run software applications that can extend their built-in functionality” (Hannon 2007). The typical contemporary features of handheld devices include the following: • Localized software applications • Web/extranet applications Project Type Applicable AMG Method Asphalt mill and pave with ride improvements; asphalt mill and pave with cross-slope or drainage corrections (hard tie-ins only at start and end) 2D sonic averaging for milling and paving Asphalt mill and pave with cross-slope or drainage corrections (tie to hard surface or curb and gutter) 3D profile milling with constant depth or 2D paving; alternately, constant depth milling and 3D paving Concrete overlay with or without ride and drainage corrections 3D paving Reclamation; shoulder and side slope widening or improvements; lane widening Grading, fine grading, base, and paving Reconstruction Grading, fine grading, base, and paving; possibly 3D profile milling and excavation New construction Grading, fine grading, base, and paving; excavation Source: Maier et al. 2018. Table 2.8. Applicable AMG methods associated with typical project types.

26 Highway Infrastructure Inspection Practices for the Digital Age • Mobile phone capability • GIS/GPS capability • Sound recording • Handwriting recording and recognition • Personal information management • Text messaging • Camera • Cellular radio/walkie-talkie Some handheld devices such as smartphones or tablets typically use satellite network posi- tioning (e.g., GNSS) to track the user’s location, and then use the inertial movement of the device to change the view as the device is moved around the environment (Mallela et al. 2020). Many state DOTs have used handheld devices for their highway infrastructure inspections. For example, the Wisconsin DOT provides its inspectors with handheld devices for measuring the placement of work and recording quantities quickly and accurately. By using handheld technology, inspectors no longer need to carry tape measures and other tools to do the same inspections. Tablet devices issued by the Wisconsin DOT include an application called OnStation that helps inspectors track the station and offsets from a particular element of a project with a high degree of accuracy in a matter of seconds (Harper et al. 2019). Similarly, the New York State DOT provides inspectors with handheld devices to measure quantities for inspection purposes. The New York State DOT noted that training inspectors to use handheld devices and other technolo- gies has helped field operations become more efficient and inspections become more accurate (Harper et al. 2019). The typical benefits of using handheld devices in highway infrastructure inspection include: • Better organization of field-generated data; • Reduced cycle time to obtain the data (e.g., one-time handling of data in the field); • Elimination of illogical data entries; • Improved accuracy for material delivery, tracking, and disposition; and • Enhanced electronic documentation and digital inspection. Recently, handheld devices have increasingly been used to display augmented reality (AR) applications because of advances in camera and display capabilities of smartphones and tablets. Using the back-facing cameras on devices to capture video of the real-world environment, AR applications can align and render virtual data and display the image on the screen. Addition- ally, handheld AR devices, such as tablets or smartphones, typically have cameras with larger fields of view on the back of the devices that are essential when the devices are held at arm’s length (Mallela et al. 2020). Another advantage of using handheld AR devices is that they allow multiple users to concurrently see the same view on multiple devices. Some vendors have focused their efforts on handheld, mobile devices for customized AR applications for construc- tion (Mallela et al. 2020). Figure 2.9 shows an example of a handheld AR device being used on a construction site. Virtual Reality and Augmented Reality Virtual reality (VR) and augmented reality (AR) are different, although they are often dis- cussed in tandem. VR completely replaces the visual world experienced by the user (Azuma et al. 2001). Typically, VR headsets place users in a computer-generated, three-dimensional, immersive environment with which they can interact. VR has been used within the construc- tion industry for many different applications such as design and collaborative visualization, and as a tool to improve construction processes. VR has been an effective tool for building design, as it provides a 3D visualization that can be manipulated in real time and used collaboratively

Literature Review 27   to explore different stages of the construction process (Whyte and Nikolic 2018). In contrast to VR, AR creates an environment in which a user has a superimposed, computer-generated view of a real-world scene. AR consists of a live, imitative version of the real world, with the capacity to add certain elements to the simulated landscape. AR preserves the user’s awareness of the real environment by compositing the real world and the virtual contents (Azuma et al. 2001). Figure 2.10 shows a reality–virtuality continuum. A VR/AR system, which typically consists of hardware components, software, and algorithms, has various applications in the construction industry. Rankohi and Waugh (2013) showed that field workers and project managers have high interest in using VR/AR technologies during the project construction phase mainly to monitor progress and detect defects or clashes in the placement of work. Shin and Dunston (2008) discussed the potential of AR applications in eight work tasks of a construction project: layout, excavation, positioning, inspection, coordi- nation, supervision, commenting, and strategizing. VR/AR technology can also be integrated with BIM to create a seamless interaction between the design and construction work (Pistorius 2017). Researchers show that VR/AR allows users to find the differences between an as-designed 3D model and an as-built facility (Georgel et al. 2007). Furthermore, other researchers imple- mented a system for visualizing performance metrics to represent progress deviations through the superimposition of 4D, as-planned models over real, time-lapsed jobsite photographs (Golparvar-Fard et al. 2009a). It is important to note that, although AR technologies have a number of applications that improve construction performance and have received considerable attention within Figure 2.9. Handheld AR tablet device with GNSS tracking hardware (Source: Mallela et al. 2020). Figure 2.10. Reality–virtuality continuum (Source: Mallela et al. 2020).

28 Highway Infrastructure Inspection Practices for the Digital Age research communities, they are still relatively new in the transportation construction industry (Mallela et  al. 2020). In fact, Pistorius (2017) indicated that the construction industry has so far only “dabbled in the use of VR/AR to aid construction projects.” However, as cheaper and higher quality 3D options come on the market, many expect that the use of VR/AR technologies for construction projects will rapidly increase in the near future (Pistorius 2017). A recent FHWA study related to leveraging AR for highway construction found that AR is able to facilitate construction inspection and review, quality assurance (QA), training, and improved project management by augmenting traditional 2D drawings with digital 3D images (Mallela et al. 2020). Several studies identified possibilities for field measurements using AR devices. For example, Moreu et al. (2018) demonstrated the use of AR devices to collect inspec- tion data in the field by capturing linear and surface information and evaluating the potential precision of the measurement process. Mallela et al. (2020) summarize the applications of AR technology in highway inspection as follows: • AR-guided site inspection—AR is able to provide inspectors with necessary information in an appropriate format as well as locations to inspect. • Remote inspection—AR is able to capture inspection data remotely. • 4D inspection—AR has the capacity to visualize inspection needs on the basis of the project schedule. • Automated inspection—The inspection of systems/components during construction can be automated by integrating AR technologies and machine learning. Continuous evolvement and improvements of mobile computing technologies such as iPads and Android tablets, software technologies, more powerful processors, smaller storage devices, higher-quality displays, and wider availability of third- party application software, have made it possible for these devices to become standalone systems with powerful functional capabilities. Because of their high mobility characteristics due to their size and weight, these mobile devices can be used in the highway construction field to perform various tasks, including recording of inspection data (Valdes and Perdomo 2013). Nondestructive Testing Technologies Nondestructive testing (NDT) technologies have been shown to have potential for use in the QA process for inspection of highway infrastructure during construction and mainte- nance of assets. Azari (2020) highlighted that “NDT technologies can assess product proper- ties and uniformity in real time as construction progresses; identify potential defects during construction, allowing for quick corrective actions; and inspection or testing at a higher sampling frequency to supplement coring and other destructive testing.” A wide range of NDT techniques and technologies have been explored for highway infrastructure inspection. A detailed description of the NDT techniques and technologies can be found in NCHRP Report 626 (Von Quintus et al. 2009) and SHRP 2 Report S2-R06-RW (Wimsatt et al. 2009). It is noted that various NDT technologies are available in handheld devices. The following sections dis- cuss briefly two emerging NDT technologies: ground-penetrating radar (GPR) and infrared thermography.

Literature Review 29   Ground-Penetrating Radar Ground-penetrating radar (GPR) is a noninvasive sensing technique that has increasingly been used for highway infrastructure inspection because of its flexibility and high potential for accurately capturing images of structures and materials. GPR uses radio waves to penetrate the structures and materials (e.g., pavement). When traveling through the structures, the radio waves create echoes at the boundaries of dissimilar materials, which are measured by the dielec- tric values. Figure 2.11 shows an example of a GPR instrument. Using a pulse echo method, GPR technology allows for high-speed data collection, typically providing results immediately in the form of dielectric values. The GPR data collection process has improved steadily over the years because of advancements in technologies. However, the interpretation process of dielectric values is challenging and often requires technical skills for calibrating and maintaining the equipment as well as the data interpretation software. GPR has a wide range of highway construction applications such as locating underground utilities, mines, caves, tunnels, or other unseen objects without excavation or destruction. For highway infrastructure inspection, GPR can be used for the following inspection activities (Goulias and Scott 2015): • Thickness and void size detection; • Cracking and delamination detection; • Corrosion detection; • Rebar location, depth, and orientation; • Density monitoring (e.g., detection of areas of segregation and poor joint density); and • Drainage-related issues. Goulias and Scott (2015) highlighted the main benefits of GPR as follows: • Higher precision and accuracy of the condition assessment of key infrastructure components and materials, • Improved speed of condition assessment, • Reduced monitoring time and cost, • Increased accuracy of the specific locations where failures occur, • Improved overall condition assessment methods and more accurate performance and life cycle predictions, and Figure 2.11. An example of GPR (Source: Torres et al. 2018).

30 Highway Infrastructure Inspection Practices for the Digital Age • Facilitated method of nondestructive testing for QA and QC activities and forensic investigations. The typical limitations of using GPR for highway infrastructure inspection include the following: • Difficulties in interpreting and analyzing GPR data (e.g., GPR data analysis cannot discrimi- nate between layers with similar dielectrics), • Challenges in estimating depth of air voids, • Limited test results (GPR cannot provide any information on mechanical properties), • Needing validations from other NDT methods, and • Limited capability in areas with high water tables (Wimsatt et al. 2009). Infrared Thermal Profilers Typically, the main components of an effective thermal profiling system include thermo- couple sensors, data acquisition loggers, thermal barriers, and temperate profiling software. Additional sensors can be placed on equipment such as the compactor or the back of dump trucks. Several state DOTs (e.g., Washington, Colorado, and Minnesota) have used infrared thermal profilers for hot mix asphalt (HMA) pavement projects. Profilers use infrared sensors to test the entire surface area of the asphalt pavement. Infrared sensors map temperature contours over the surface of the material, and the contours are used to evaluate materials by indicat- ing their surface temperatures and variations in surface temperature (SHRP2 R06C 2018). Figure 2.12 shows an example of a thermal profiler along with temperature readings of 193oF to 200oF. The thermal profile can show variability in temperatures between trucks. Variations from the ideal value indicate a potential problem or unacceptable quality. For example, by scanning any cold spots or streaks in the pavement layers, profilers create a live thermal profile of the asphalt mat. The cold spots indicate potential for fatigue cracks, raveling, or potholes. Figure 2.12. An example of a thermal profiler (Source: Adapted from SHRP 2 R06C 2018).

Literature Review 31   The thermal streaks indicate potential for longitudinal cracking and raveling. Segregation, a critical issue for pavement performance, can be effectively identified and eliminated by using the thermal profiler. It is noted that a thermal profiler is often used in combination with IC to improve asphalt paving quality. The main benefits of using a thermal profiler for highway infrastructure inspection include the following: • Evaluating mat uniformity through temperature uniformity, • Providing visual data that are easy to interpret (e.g., uniform temperatures imply uniform densities, which usually mean lower maintenance), • Allowing contractors to detect temperature segregation problems in real time and make adjustments during construction, • Reducing future distress and maintenance costs, • Reducing dispute resolution, and • Increasing communication between material plants and pavers to minimize risk (SHRP2 R06C 2018). Summary The literature review results presented in this chapter document the most relevant topics in the use of technologies for highway infrastructure inspection. This chapter provides key infor- mation for understanding the state of practice by DOTs of using various technologies to inspect highway infrastructure during construction and maintenance of assets. The key concepts, typical applications, benefits, and challenges related to highway inspection of the sample technolo- gies within four main technology areas of (1) geospatial technologies, (2) remote sensing and monitoring technologies, (3) mobile devices and software applications, and (4) nondestructive evaluation methods are discussed in detail. For the geospatial technology area, the GNSS/GPS, GIS, UAS, RTS, photogrammetry, and e-ticketing technologies were discussed with respect to their use for highway infrastructure inspec- tion. The typical applications of the geospatial technologies in highway infrastructure inspection include earthwork, paving, verification, as-built surveys, site and progress monitoring, quality assurance and quality control, and asset management. For the remote sensing and monitoring technology area, LiDAR and 3D laser scanning, RFID, IC, and barcodes and readers were discussed with respect to their use for highway infra- structure inspection. The LiDAR system can be used for a wide range of applications including earthwork, paving, roadway design, AMG and control, quality assurance and quality control, and asset management. The main applications of RFID for highway infrastructure inspection include tracking PCC test specimens, monitoring in-place PCC, tracking construction materials, and locating buried infrastructure. The IC technology can be applied to all pavement layer materials and soils aggregates to improve the quality, uniformity, and long-lasting performance of pavements. Barcodes can be used to automate material tracking systems and eliminate manual data entry. For the mobile devices and software applications area, 3D engineered models, AMG, hand- held data collectors, and VR/AR were discussed with respect to their use for highway infrastructure inspection. The 3D engineered model technology enables faster, more accurate, and more efficient planning and construction. AMG is a suitable tool for projects that involve large amounts of earthwork/paving or new alignments, or projects that require accurate digital terrain models. The main benefits of using handheld devices in highway infrastructure inspection include better organized field-generated data, reduced cycle time to obtain the data, improved

32 Highway Infrastructure Inspection Practices for the Digital Age accuracy for material delivery, and enhanced electronic documentation and digital inspection. The main applications of VR/AR technology in highway inspection include AR-guided site inspection, remote inspection, 4D inspection, and automated inspection. For the nondestructive evaluation methods area, GPR and infrared thermography were discussed with respect to their use for highway infrastructure inspection. GPR has a wide range of highway construction applications such as locating underground utilities, mines, caves, tunnels, or other unseen objects without excavation or destruction. Infrared thermo g- raphy uses sensors to evaluate materials by measuring their surface temperatures and variations in surface temperature. Variations from the ideal value indicate a potential problem or unaccept- able quality.