Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
NCHRP 07â19(02) Final Report 17 Chapter 2: Research Approach This section describes the approach used for testing the nonmotorized counting devices and technologies investigated in Phase 2. Chapter 3 presents the results of this testing and merges the results with those obtained during Phase 1. Consistent with NCHRP practice, no product names or manufacturers are described here by name. Rather, when more than one product was tested representing a particular technology, the products are referred to as âProduct A,â âProduct B,â etc. Furthermore, the objective of testing different products was not to rate individual products, but rather to identify whether different implementations of a particular technology appear to have a bearing on the observed count error. TECHNOLOGY TESTING OBJECTIVES The primary objective of testing the counters used in this research was to determine their accuracy under a variety of conditions. Good measurements of accuracy can be used to develop correction factors to account for regular underâ or overâcounting by specific sensor technologies under specific conditions. One of the most common sources of error for devices that detect users from a distance is the tendency to undercount due to occlusion, or one traveler blocking another from the counting deviceâs field of detection. Devices that detect users through contact or closeâproximity movement (e.g., pneumatic tubes and loop detectors) may also have other sources of error, such as not detecting enough pressure, not detecting certain materials, or counting motor vehicles in addition to bicyclists. Other counting device characteristics were also evaluated, including: ï· Ease of installation, ï· Labor requirements, ï· Security, ï· Maintenance requirements, ï· Software requirements, ï· System power requirements, ï· Impact of weather conditions, ï· Cost (e.g., purchase, installation, and other costs related to obtaining and activating devices), ï· Coverage area (versatility), and ï· Flexibility to use data outputs for various applications, including matching the Traffic Monitoring Guide format (FHWA 2013).
NCHRP 07â19(02) Final Report 18 SENSOR TECHNOLOGY SELECTION Two of the devices included in Phase 2 entered the market for practical pedestrian and bicycle volume data collection after Phase 1 was underway and represented technologies that had not been tested in Phase 1. These devices were: ï· Thermal imaging camera. These devices combine the technologies of passive infrared detection and automated counting from imaging. The camera detects the infrared radiation (i.e., heat) given off by pedestrians and bicyclists, and the system counts the number of heatâ emitting objects that pass through a defined detection zone within the cameraâs field of view. ï· Radar. These devices operate by emitting electromagnetic pulses and deducing information about the surroundings based on the reflected pulses. The device that was tested is designed to be buried in the pavement. Both of these sensors required additional components to create a usable counting package. The thermal imaging camera, which was mounted on a street light mast arm, required an interface card installed in a nearby traffic signal controller cabinet to process and store the data, as well as data and power cabling. The radar unit, which was buried in a bicycle lane, required two additional communications devices. The first was a poleâmounted repeater unit located nearby that received data transmitted by the buried sensor. The repeater retransmitted the data to a receiver module which processed the data and uploaded it to the vendor via a cellular modem. Both devices also required vendorâsupplied software for viewing and exporting the count data. The other three counting technologies, bicycleâspecific pneumatic tubes, piezoelectric strips, and passive infrared, had been tested in Phase 1, but using products from other vendors. The bicycleâ specific pneumatic tubes tested in Phase 1 experienced problems with motor vehicles dislodging the tubes; as a result, few data were collected for this technology during Phase 1 relative to other technologies. Two piezoelectric products from different vendors had been planned to be tested in Phase 1, but one product experienced problems with vendor support and was therefore removed from the Phase 1 research program. The passive infrared counter that was tested in Phase 2 was part of a combination counter that included a piezoelectric sensor for counting bicyclists. Including devices from different vendors that share a common sensing technology allows the research to investigate whether issues with accuracy and precision appear to be inherent to the technology, or are a result of a particular vendorâs implementation of the technology. SITE SELECTION Site Selection Process Three jurisdictions were selected for testing the devices on the basis of (1) the researchers having existing staff contacts at the jurisdictions who could facilitate obtaining any required permits to install the devices, and (2) the locations being close to the vendors of the devices, allowing easier coordination for installation, calibration, and (if necessary) troubleshooting. These jurisdictions were:
NCHRP 07â19(02) Final Report 19 ï· Arlington County, Virginia; ï· Washington, D.C.; and ï· Oakland, California. The site used in Arlington was the same test site on the Four Mile Run multiuse trail used in Phase 1. The site was selected as both piezoelectric counters being tested were already located there, one having been installed by Arlington County in April 2010 prior to Phase 1 and the other having been installed during Phase 1. The test sites in the other two cities were selected in consultation with local agency staff and the vendor on the basis of: ï· From the researchersâ point of view, locations with relatively high bicycle volumes, to allow the devicesâ accuracy and precision to be tested under a range of volume conditions; ï· From the jurisdictionsâ point of view, locations that would provide useful data for them in the future, as they would take over ownership of the counter following the tests; and ï· From the vendorsâ point of view, locations that complied with the vendorsâ guidance for suitable installation locations for the sensors and their supporting technology (e.g., detection area dimensions, access to power, radio communication ranges). The bicycleâspecific pneumatic tubes, being portable, were tested during a oneâweek period at each test site. Individual Site Descriptions This section describes the sites used for the followâup testing. Descriptions of the sites used in Phase 1 are provided in NCHRP WebâOnly Document 205 (Ryus et al. 2014b). Arlington, Virginia The Arlington site was located on Four Mile Run Trail east of Iâ395 (Figures 2â1 and 2â2). Two piezoelectric counters were already located there. One had been installed by Arlington County in April 2010, prior to Phase 1 (not visible in Figure 2â2), while the other was installed during Phase 1 (marked âAâ in Figure 2â2). The latter device experienced communication problems which the deviceâs original vendor was unable to solve; therefore, the device could not be tested during the original research. However, at the start of Phase 2, Arlington County staff worked with the deviceâs new vendor to install new hardware to connect to the existing piezoelectric sensors and to obtain updated software; these actions solved the communications problem and allowed the device to be tested. In addition, the new vendor installed a passive infrared sensor (marked âBâ in Figure 2â2) that allowed pedestrians to be detected in addition to bicyclists. (The piezoelectric sensor detects bicycles only, the passive infrared sensor detects both pedestrians and bicycles, and the difference in the two counts is the pedestrian count.) Bicycleâspecific pneumatic tubes were also temporarily installed at this site for one week (marked âDâ in Figure 2â2); the tubes were dislodged toward the end of the week and were not reinstalled. Other equipment at the site includes a passive infrared counter installed by Arlington County (marked âCâ in Figure 2â2) that was tested in Phase 1.
Figure 2â Source:  © Figure 2â 1. Arling 2016 Google. 2. Four M ton and Wa ile Run Tra shington, D il (Ground V 20 .C. Sites (Ae iew) N rial View) CHRP 07â19 (02) Final Report
Washing The Wash just south high bicy placed on desired d installed the came system su would ma testing pe had been field of vi Bicycleâsp oneâweek traffic), th 15th Stre Figure 2â ton, D.C. ington site w of Massach cle volumes a street ligh etection are in a traffic si ra by runnin pports remo nually retri riod. The Di identified du ew. ecific pneum testing peri e tubes exte et NW. 3. 15th S as located usetts Avenu and proximi t mast arm a a on the cycl gnal control g cables thro te access, gi eve the data strict Depar ring the site atic tubes w ods. To simu nded across treet NW (G on a bidirect e (Figures 2 ty to a traffic nd fitted wi e track (Figu ler cabinet o ugh existing ven the shor from the int tment of Tra selection p ere tempor late their in the bicycle round View 21 ional cycle tr â1 and 2â3). signal contr th a lens tha re 2â4). The n the north s conduit und t duration o erface card i nsportation rocess as po arily installe tended use ( track and the ) N ack on the w The site was oller cabine t provided a deviceâs req ide of the in er the pave f the test, it w n the field fo trimmed bra tentially inte d at this site counting bic two southb CHRP 07â19 est side of 1 selected for t. The therm good field o uired interfa tersection a ment. Althou as decided llowing each nches on a s rfering with during the ycles and ca ound motor (02) Final R 5th Street N its relativel al camera w f view of the ce card was nd connecte gh the coun that the ven oneâweek treet tree th the cameraâ first of the tw rs in mixed vehicle lane eport W y as d to t dor at s o s on
Figure 2â Oakland, The Oakla Terrace ( sidewalk range rad included the inform sensor to data to th Bicycleâsp use (coun lane, and tubes we standard 4. 15th S California nd site was Figures 2â5 a . The radar s io, was buri an access po ation receiv the access p e vendor. ecific pneum ting bicycle the two wes re dislodged pneumatic t treet NW (T located in a nd 2â6). On ensor unit, w ed in the pav int in the tra ed from the oint, and a c atic tubes w s and cars in tbound moto within the f ubes for the hermal Cam bicycle lane âstreet parki hich include ement in the ffic signal co sensor, a po ellular mode ere tempor mixed traffi r vehicle lan irst 24 hours remainder o 22 era Field o on westboun ng was locat s a battery r middle of t ntroller cab leâmounted m connecte arily installe c), the tubes es on Grand following in f the test we N f View) d Grand Av ed between ated for 10 he bicycle la inet at the n repeater tha d to the acce d at this site extended ac Avenue. Th stallation an ek. CHRP 07â19 enue just we the bicycle l years of use ne. The com earby inters t relayed da ss point that . To simulat ross the par e thinner bic d were repl (02) Final R st of Park Vi ane and the and a shortâ plete system ection to pro ta from the uploaded co e their inten king lane, bi ycleâspecifi aced with eport ew cess unt ded cycle c
Figure 2â Source:  © Figure 2â 5. Oakla 2016 Google. 6. Oakla nd Site (Aer nd Site (Gro ial View) und View) 23 NCHRP 07â19(02) Final Report
NCHRP 07â19(02) Final Report 24 SUMMARY OF TEST SITES AND TECHNOLOGIES Table 2â1 summarizes the counting technologies and products tested at each site. Table 2â1. Counting Technologies and Products Tested in Phase 2 Technology Category Product Site Arlington Washington, D.C. Oakland Piezoelectric strips A Existing B Existing Passive infrared C New Pneumatic tubes: bikeâspecific C New New New Pneumatic tubes: standard C New Thermal camera A New Radar A New Notes: Existing = existing installation, New = new device installation during Phase 2. Product letter designations are continued from Phase 1 testing (e.g., passive infrared products A and B were tested during Phase 1 but are not shown in this table). EVALUATION METHOD Evaluation Criteria Pedestrian and bicycle counting technologies were evaluated according to several performance measures. The primary evaluation criterion was accuracy. However, ease of installation, labor requirements, security, maintenance requirements, software requirements, cost, flexibility of data, and other characteristics were also assessed. Accuracy Accuracy was determined by comparing data generated by the automated counting devices with ground truth pedestrian or bicycle volumes generated from manual counts taken from video recordings. This method of generating ground truth counts was selected primarily because it is believed to be the most accurate, as the data reducers can play back the timeâsynchronized videotape at a suitable speed for making sure all pedestrians or bicyclists are recorded and classified to the appropriate time period. Data reducers can also rewind the tape if necessary to check that no pedestrian or bicyclist was missed. This method also had the benefit of allowing a relatively large amount of video to be collected, from which the researchers could select only the time periods with environmental conditions (e.g., rain, darkness) or nonmotorized volumes of interest to be counted. Finally, the method allowed data to be collected at a different time from when it was reduced, which helped spread out the use of research team labor. The process used to generate the ground truth counts is described in a subsequent section.
NCHRP 07â19(02) Final Report 25 All of the counters tested, except for pneumatic tubes, were permanent installations. The tube counter was moved from site to site and only installed during a period when a camera was present collecting video for the ground truth counts. Ground truth video for the permanent installations was recorded during two oneâweek time periods. The first occurred after the devices had been calibrated and all parts of the system (e.g., communications) were working. The second occurred later, when warm weather was forecast, to capture higher volumes of bicyclists and pedestrians. The pneumatic tubes were in place during the first ground truth data collection period for each site. As described in detail in Chapter 3, correction factors were developed by comparing the ground truth counts to the counts produced by each device that was tested. The researchers selected specific hours from each week of video to develop ground truth counts, corresponding to hours when specific conditions of interest occurred (e.g., high bicycle volumes, rain). Other Characteristics Evaluated In addition to accuracy, the following factors were evaluated: ï· Ease of installation. Potential installation difficulties include the need for specialized equipment (e.g., saw cutters, junction boxes), the need for specialized knowledge, difficultâ toâreach mounting points (e.g., points high above the ground), difficulties with calibration, and length of time required to complete the installation under normal conditions. The level of technical assistance required from vendors to install and calibrate the technologies was noted (i.e., how easy was the technology to install âout of the boxâ?). ï· Labor requirements. Labor requirements include the time and effort needed for installation, ongoing maintenance, data cleaning, and data analysis. ï· Durability and security from theft and tampering. Durability and security were evaluated based on both inspection of the hardware to assess the relative durability and resistance to tampering or theft, as well as recording any damage sustained while a device was in place. Some possible problems that were anticipated included theft, obstruction of optical devices (lenses) with foreign objects (e.g., chewing gum, paint), disconnection of parts or wiring (e.g., pneumatic tubes removed from logger), and opened device cases. ï· Maintenance requirements. Maintenance requirements included regular site visits for visual inspection, data downloads, reâcalibration of the device to maintain accuracy, optical lens cleaning, or removal of obstructions. Anything requiring a site visit after installation to continue accurate data collection was documented as a potential maintenance requirement. Costs associated with ongoing maintenance were considered. ï· Flexibility of data produced. The data formats exported by the counter were evaluated for ease of export and transfer, as well as compatibility with the FHWAâs Traffic Monitoring Guide format (FHWA 2013). ï· Cost. Cost refers to the monetary cost of purchasing and installing the device, including any additional accessories or software necessary for count data collection and analysis, as well
NCHRP 07â19(02) Final Report 26 as costs associated with ongoing use, such as replacing batteries and charges associated with telemetry. ï· Weather tolerance. To the extent scheduled data collection times and budget permitted, technologies were evaluated based on their ability to operate and maintain accuracy under a range of climates and weather conditions (e.g., heat, cold, rain, fog). These factors were evaluated largely from a qualitative standpoint, except for in cases where a quantitative approach was feasible (e.g., times and costs). VIDEO DATA REDUCTION PROCESS TO GENERATE GROUND TRUTH COUNTS All of the manual counts were conducted using video footage collected at the test sites. Videos were typically recorded for two sevenâday periods at each of the sites in the study, aiming to test the technologies under a diverse set of environmental and traffic conditions. The video cameras and counting devices were timeâsynchronized with each other to ensure accurate assessment of events captured by specific time periods. Videos were shipped from the test sites on DVDs. The video collection dates for the followâup research are listed in Table 2â2. The first week of data collection occurred soon after device calibration had been completed. The second week of data collection occurred during a week when good weather was forecast, in an effort to maximize the number of bicyclists observed. An additional day of testing was added in Oakland to coincide with Bike to Work Day, when high bicycle volumes were expected. Table 2â2. Dates of Video Collection for All Sites Used in Phase 2 Site Video Data Collection Dates Arlington June 19â26, 2015 October 19â28, 2015 Washington, DC February 22â29, 2016 April 18â23, 2016 Oakland November 10â15, 2015 April 18â24, 2016 May 12, 2016 INTERRATER RELIABILITY Similar to the process used in Phase 1, interrater reliability was evaluated by having data collectors conduct counts on a set of 30 oneâminute video clips. These counts were then evaluated against one another using the Concordance Correlation Coefficient, a measure of agreement commonly used for interrater reliability evaluation. All data collectors largely agreed with one another, producing very high rates of interrater reliability, as shown in Table 2â3. Note that in Phase 1, an error was made in the interrater reliability data entered for one of the data collectors and subsequently reported in the final report; the results in this report have been corrected. Table 2â3 includes all of the data collectors who participated in either Phase 1 or Phase 2.
NCHRP 07â19(02) Final Report 27 Table 2â3.  Results of Interrater Reliability Evaluation (Concordance Correlation Coefficients): Phases 1 and 2 Combined   Data Collector   1 2 3 4 5 6 7 8 9 Da ta  Co lle ct or  1 1.000 0.989 0.986 0.989 0.989 0.989 0.989 0.986 0.992 2 âââ 1.000 0.997 0.989 0.995 0.995 0.995 0.997 0.992 3 âââ âââ 1.000 0.987 0.992 0.992 0.992 0.995 0.989 4 âââ âââ âââ 1.000 0.995 0.995 0.995 0.992 0.997 5 âââ âââ âââ âââ 1.000 0.995 1.000 0.997 0.997 6 âââ âââ âââ âââ âââ 1.000 0.995 0.997 0.997 7 âââ âââ âââ âââ âââ âââ 1.000 0.997 0.997  8 âââ âââ âââ âââ âââ âââ âââ 1.000 0.995  9 âââ âââ âââ âââ âââ âââ âââ âââ 1.000 DATA STORAGE For Phase 2, a Google Form was used by data collectors to input their manual count data. Automated counts and weather data files were all stored in CSV files, which were merged using the Python programming language. Code is available upon request from SafeTREC for any research team that would like to duplicate the methodology. WEATHER DATA SOURCES To test the effects of weather on accuracy, weather data were obtained from the National Climatic Data Centerâs Quality Controlled Local Climatological Database, except for weather data for Montreal, which was acquired through the Government of Canadaâs Historical Climate Database. Data was obtained for the nearest monitoring station for each city, as detailed in Table 2â4. This table includes all of the weather stations used for Phases 1 and 2. Table 2â4. Weather Stations Used as Sources of Weather Data for Equipment Testing City Weather Station Weather Station ID Washington, DC and Arlington, VA Ronald Reagan International Airport 13743/DCA San Francisco, CA San Francisco International Airport 23234/SFO Berkeley and Oakland, CA Oakland International Airport 23230/OAK Davis, CA Nut Tree Airport (Vacaville, CA) 93241/VCB Portland, OR Portland International Airport 24229/PDX Minneapolis, MN MinneapolisâSt. Paul International Airport 14922/MSP Montreal, QC, Canada Montreal International Airport not applicableÂ
NCHRP 07â19(02) Final Report 28 DATAÂ CLEANINGÂ The automated counters did not export data in a consistent format. Some devices exported counts in 15âminute bins, some exported counts in 1âhour bins, and some timestamped each individual count. The timestampâstored values were converted to 15âminute bins to allow for consistent analysis. All data were then converted to 1âhour time periods for consistency of analysis. For the devices that output data in 15âminute intervals, this meant summing the automated and manual count values across the four component intervals of the hour. For the devices that output data in 1â hour format, the manual count values were summed across the four component intervals to yield an hourly total. Some devices being tested performed in ways inconsistent with other devices of the same type installed at different sites. While every effort was made not to omit any data points without a solid rationale to justify their exclusion, the sites summarized in Table 2â5 were excluded for the reasons shown. The research team felt that these periods of data collection were not representative of typical operating conditions of the technologies being tested (given their inconsistency with other devices of the same type). In some cases, the source of the problem could not be diagnosed, but possible explanations include a problem inherent to the device, installation error, calibration error, or poor placement of the ground truth video camera.
NCHRP 07â19(02) Final Report 29 Table 2â5. Summary of Data Removed from Analysis in Phases 1 and 2 Research Phase Site and Device Reason Phase 2 Oakland radar A single outlier observation was removed due to strong effect on results. Results in Chapter 3 are presented both with and without this observation. Phase 1 L Street inductive loops Very high undiagnosed overcounts, inconsistent with performance at all other sites. L Street passive infrared High overcounts during one month of testing were removed. These were attributed to the installation of the counter, which was oriented toward a plate glass window with foliage in the foreground. During the summer, when temperatures were high, the foliage was suspected to heat up and trigger false positives. Rue Maisonneuve inductive loops Only 4 hours of observations were taken, 2 of which were false 0s, leading to limited reliability of results. University Ave. inductive loop facility counts Data at this location were collected before the need to collect counts for the full facility was identified. The results for the âdetection zoneâ are reported on. Eastbank Esplanade radio beam Automated counts were faulty, with extended periods of 0 counts followed by periods averaging 1000s of people per hour. Berkeley radio beam Automated counts appeared to be faulty, with extended periods of 0 counts during the middle of the day. Midtown Greenway inductive loops (sensor counts only) Poor ground truth camera placement that prevented researchers from defining whether bicyclists were in or out of the loopâs detection zone, which did not cover the entire width of the facility. Midtown Greenway pneumatic tubes These pneumatic tubes showed substantial undercounting when initially installed. A representative from the vendor adjusted the sensitivity setting remotely, after which the counter demonstrated a consistent pattern of overcounting. The consistent pattern of overcounting suggests that the tube could be adjusted again to achieve a more accurate count, but because the research team did not recognize that this was needed until after video had been collected, this counter was removed from the analysis to avoid an unfair representation of the device that would be avoided in practice by more careful calibration of the device. 15th Avenue (Minneapolis) pneumatic tubes These tubes substantially overcounted bicyclists, inconsistent with pneumatic tubes at other sites. Part of this could be attributable to poor validation video footage, as trucks and buses frequently obscured the detection zone from the camera.Â
NCHRP 07â19(02) Final Report 30 SUMMARY OF DATA COLLECTED AND EVALUATED Video data were collected twice for most of the sites in the study. Data collection periods were typically between 2 and 5 days long (Phase 1) or 7 days long (Phase 2), and took place between March and November 2013 for Phase 1 and between June 2015 and May 2016 for Phase 2, as described earlier in this chapter in Table 2â2. Specific hourly intervals were selected for reduction from these large batches of video. Videos were selected heuristically, with a focus on maximizing the amount of data under extreme conditions (e.g., high volumes, hot or cold temperatures, thunderstorms) and maximizing the amount of data for the technologies that were underrepresented in the study (active infrared, piezoelectric strips, and radio beam). Table 2â6 summarizes the characteristics of the collected data. All of the variables denoted âhoursâ show the number of hours of automated counts with corresponding ground truth counts under each of those conditions. The variables marked with (mean/SD) show the mean value and standard deviation of the variable in question across all hours of video used for analysis for a given technology. Table 2â6. Summary of All Data Evaluated in Phases 1 and 2  Notes: SD = standard deviation. âFacilityâ considers the ground truth count to be all bicyclists on the facility (i.e., street or path) to account for bypass errors. Condition Active Infrared Thermal Camera Passive Infrared Radar Radio Beam Induction Loops Induction Loops (facility) Piezoâ electric Strips Pneuâ matic Tubes Total hours of data 34 28 398 32 56 165 202 120 279 Temperature (°F) (mean/SD) 64 / 25 54 / 12 72 / 11 60 / 5 78 / 6 70 / 13 69 / 15 73 / 11 69 / 10 Temperature range  (°F) 12â86 40â83 13â93 48â72 66â90 17â93 12â93 48â93 40â91 Hourly user volume (mean/SD) 327 / 209 100 / 97 258 / 190 69 / 50 271 / 186 152 / 104 184 / 163 104 / 53 160 / 171 Hourly volume range 28â769 8â340 1â846 1â205 18â769 7â500 1â731 21â283 7â963 Nighttime hours 4 5 35 2 0 23 28 4 22 Rain hours 0 10 31 3 0 5 3 3 33 Cold hours (<30 °F) 6 0 7 0 0 3 9 0 0 Hot hours (>90 °F) 0 0 15 0 0 4 4 10 3 Thunder hours 0 1 8 0 0 1 1 3 4Â