Exhaust Emissions from In-Use General Aviation Aircraft (2016)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

20 Exhaust Emissions from In-Use General Aviation Aircraft In Table 3-2 confidence intervals are generally the best (smallest) for the Lycoming O-320 engine family because the 16 replicate tests contribute to good statistics of these inherently highly variable emissions burdens (4 partial tests were excluded). When we apply the EDMS protocol of grouping all engines at or below 200 horsepower (HP), somewhat poorer statistics for the HC burden emerge, despite a greater number of measurements. Table 3-2 shows that a better grouping of engines would be anything below or equal to 160 HP. Although some of these confidence intervals are greater than 100%, emission burdens may never be negative. In Table 3-3, selected pre-existing results from FAEED, EDMS and FOCA are shown. These results are not reported with any confidence intervals and, at least for the FOCA data, are the result of a single aircraft measurement. Data is considered valid if it falls within the 95% confi- dence intervals shown above. The General Electric CF34 jet is compared to ICAO data, although there is only a single aircraft measurement. For this jet, strict confidence intervals of ±50% were assumed. The following invalid existing data points are evident from Table 3-3: 1. The FOCA Lycoming O-320 HC emissions burden is too low (69 vs 258 g HC/LTO) 2. The FOCA Lycoming IO-360 CO emissions burden is too high (6988 vs 4387 g CO/LTO) 3. The FOCA Lycoming O-540 NOx emissions burden is too low (3 vs 22 g NOx/LTO) 4. The ICAO General Electric CF34-3A1 CO emissions burden is too low (3350 vs 7315 g CO/LTO) 5. The EDMS Lycoming O-320 HC emissions burden is too low (115 vs 258 g HC/LTO) Of these invalid data, Item 5 is the most important. Item 5 involves an underestimate of HC emissions in EDMS/AEDT on the common Lycoming O-320 engine and, as such, will probably be used by default in any calculation GA airport emissions. The emissions are underestimated by a factor of 2.3. Given the large variability in the emissions performance found in piston engines, it is unlikely that any of the cases found to be invalid via the comparison criteria adopted here were flawed measurements. The research results suggest that the characterization work was probably legitimate and that there is just a significant amount of variability in piston engine emissions. Given the many repeat measurements of Lycoming O-320 family engines, the research team recommends average data from this family be substituted in current software for airport emis- sions estimates. Other engine families do not have the same number of replicates, and so the research team does not recommend them for substitution until additional data is available. Engine Data Source HC [g/LTO] valid? CO [g/LTO] valid? NOx [g/LTO] valid? diverse Prop-200hp FAEED162 116 YES 5350 YES 8 YES diverse Prop-300hp FAEED160 345 YES 5481 YES 20 YES tPMm [g/LTO] valid? 0.27 YES 2.02 YES Lycoming O-320 FOCA 69 NO 3426 YES 36 YES 0.26 YES Lycoming O-360 FOCA 115 YES 5948 YES 17 YES Lycoming IO-360 FOCA 168 YES 6988 NO 9 YES 0.32 YES 1.30 YES Lycoming O-540 FOCA 215 YES 8470 YES 3 NO Lycoming IO-540 FOCA 244 YES 7974 YES 23 YES 1.11 YES General Electric CF34-3A1 ICAO 313 YES 1 rep 3350 NO 1 rep 1137 YES 1 rep Lycoming IO-360 EDMS 104 YES 4017 YES 20 YES Lycoming O-320 EDMS 115 NO 5326 YES 8 YES Table 3-3. Validation of existing data. The size of the color bars is proportional to the magnitude of the emissions burden for HC (orange), CO (pink), NOx (green) and tPMm (blue).

21 Sensitivity Analysis on Airport Emissions Sensitivity Analysis Using EDMS/AEDT Tools A hypothetical GA airport’s emissions were calculated before and after inclusion of the newly measured emission indices. Differences of CO: -6%, HC: 194% and NOx: 64% were found. The effect of emissions variability is important. In fact, these seemingly large changes are not statistically different from the baseline scenario because the 95% confidence intervals for the updated scenario are very wide and the baseline scenario falls within their bounds. Introduction and Purpose This chapter summarizes the method used for and results of the sensitivity analysis comparing the effects of updating EDMS/AEDT database default emission indices with measured emission indices performed for ACRP Project 02-54, “Measuring and Understanding Emission Factors for GA Aircraft.” The sensitivity analysis consisted of using the FAA’s Emissions & Dispersion Mod- eling System (EDMS) modeling tool to determine the potential effect on computed emissions that may result from replacing or supplementing existing emissions indices within the EDMS/AEDT database with those measured in the field campaigns during ACRP Project 02-54. For this assess- ment, EDMS was used, because it produces results similar to the new AEDT. Assessment For ease of understanding, the analysis is divided into three steps: 1. GA Airport Fleet: FAA’s National Tail Number Registry was used to identify the top 50% of aircraft engines within each engine category [i.e., single engine piston (SEP), multi-engine piston (MEP) and single engine turboprop (SETP)] to develop a hypothetical national GA airport fleet. 2. Engine Substitutions: Engine substitutions were performed to use the measured data, deter- mine engine similarities, and provide recommendations on substitute engines for use in EDMS/AEDT modeling when no measured or database emission information is available. 3. Sensitivity Analysis: A sensitivity analysis was performed to compare the effects of updating EDMS/AEDT default emission indices with measured emission indices. The sensitivity analysis provides understanding of the potential effect of the research results on a hypothetical airport. Detailed discussion of these three steps follows. Step 1—Identifying a Hypothetical National GA Airport Fleet FAA’s National Tail Number Registry was used to select a representative fleet for a hypothetical airport. C H A P T E R 4

22 Exhaust Emissions from In-Use General Aviation Aircraft FAA’s National Tail Number Registry was queried to construct a representative aircraft popula- tion. The data was further summarized and ranked by number of occurrence for each aircraft engine within each engine category. The top 50% of each engine category was then used as the Hypothetical National GA Airport fleet, as shown in Table 4-1. This category break-down did not allow capture of every possible aircraft. For example, turbofan engines were excluded. The difficulty in constructing this fleet was due to the lack of data from the FAA registry about GA operations. Step 2—Engine Matching and Substitutions Engine substitutions were required because some engines are not available in the airport emissions simulation software AEDT and EDMS. Substitutions were done by comparing aircraft weight and engine horsepower. Flowcharts outline the required steps. The aircraft engines from the Hypothetical Fleet (Table 4-1) were matched to the engines sampled in each field campaign for this project only if they were exact engine matches. Engines Table 4-1. Hypothetical National GA Airport fleet. Engine Category* Rank** Cumulave Percent of Engine Category* Aircra Make Aircra Model Engine Family Total Naonal Occurrence Percent of Hypothecal Fleet SEP 1 6 CESSNA 172 O 320 10570 11 2 12 CESSNA 182 O 470 10057 10 3 18 CESSNA 150 O 200 9266 10 4 22 PIPER PA 28 O 320 7633 8 5 26 CESSNA 172 O 300 6836 7 6 29 PIPER PA 28 O&VO 360 5346 6 7 31 CIRRUS DESIGN CORP SR22 IO 550 2854 3 8 33 MOONEY M20 IO 360 2704 3 9 34 PIPER J3C 65 A&C65 2359 2 10 35 CESSNA 152 O 235 2254 2 11 37 CESSNA 180 O 470 2128 2 12 38 CESSNA 172 IO 360 2102 2 13 39 PIPER PA 28 IO 360 2075 2 14 40 PIPER PA 22 O 320 2049 2 15 41 BEECH 35 IO 520 2021 2 16 43 PIPER PA 18 O 320 1940 2 17 44 CESSNA 170 C145 1795 2 18 45 PIPER PA 32 TIO 540 1749 2 19 46 CESSNA 210 TSIO 520 1725 2 20 47 AERONCA 7AC A&C65 1711 2 21 48 BEECH 35 IO 470 1677 2 22 49 CESSNA 140 C85 1468 2 23 49 CESSNA 182 IO 540 1406 1 24 50 MOONEY M20 O&VO 360 1328 1 25 51 PIPER PA 28 O 540 1312 1

Sensitivity Analysis on Airport Emissions 23 were substituted when the engines from the Hypothetical Fleet were not present in the EDMS/ AEDT databases. In these cases, the research team used one of the following methods, based on data availability: • Simple substitution method, or • Advanced substitution method The simple substitution method was used when detailed data was not available. Substitutions were made based on aircraft and engine family. The advanced substitution method involves substituting an engine based on engine/aircraft family as well as one with similar emission coefficients, horsepower, and weight. For a more conservative analysis, an aircraft/engine with higher emission coefficients and aircraft weight is chosen. Obtaining this information requires researching the sampled aircraft/engine and look- ing up emission coefficients and aircraft weights within the EDMS/AEDT databases. Given that the hypothetical airport is constructed from the FAA Tail Registry Database, only engine family is available, and only the simple method is required. This type of engine substitution is standard practice for firms specializing in using EDMS and AEDT software. Figure 4-1 outlines the procedure for the simple substitution method. Engine/aircraft entries for the hypothetical airport were compared one by one to the available combinations in EDMS/ AEDT. Engine matches were prioritized over aircraft matches. For example, if a Cessna 150 with an O-320 engine was sampled in the field but this aircraft/engine combination was not in the EDMS/AEDT modeling databases, the recommended substitution would be a Cessna 172 with an O-320 engine, because the engine is a match, and the Cessna 172 aircraft is comparable to the Cessna 150 in terms of weight. Table 4-1. (Continued). Engine Category* Rank** Cumulave Percent of Engine Category* Aircra Make Aircra Model Engine Family Total Naonal Occurrence Percent of Hypothecal Fleet MEP 1 6 CESSNA 310 IO 470 1169 1 2 12 BEECH 95 IO 470 1068 1 3 17 PIPER PA 30 IO 320 937 1 4 22 PIPER PA 31 TIO 540 902 1 5 27 PIPER PA 23 TIO 540 899 1 6 32 PIPER PA 34 TSIO 360 883 1 7 36 CESSNA 421 GTSIO 520 716 1 8 40 CESSNA 340 TSIO 520 660 1 9 43 CESSNA 337 IO 360 649 1 10 47 PIPER PA 23 O 320 597 1 11 50 BEECH 58 IO 520 593 1 12 53 CESSNA 414 TSIO 520 565 1 SETP 1 17 CESSNA 208 PT6A 466 <0 2 33 PILATUS PC 12 PT6A 67 464 <0 3 52 EADS SOCATA TBM 700 PT6A 66 259 <0 TOTAL 97192 100% *Single engine piston (SEP), mul-engine piston (MEP) and single engine turboprop (SETP) **Rank indicates the rank by number of occurrence for each aircra engine within each engine category (e.g., SEP, MEP, and SETP).

24 Exhaust Emissions from In-Use General Aviation Aircraft A second set of matches was constructed, substituting experimental data when available. Figure 4-2 outlines this procedure. If no experimentally measured data matches exactly, no further substitution is done and the procedure defaults to the EDMS/AEDT dataset match. Table 4-2 shows sample mappings for the five most common SEP aircraft and lists the Hypo- thetical Fleet aircraft make/model and engine family, the sampled aircraft engine model, and the EDMS/AEDT aircraft make/model and engine it was matched with. Priority was given to match the aircraft engine family over the aircraft make/model. (Table L-1 in Appendix L details the full mapping for all hypothetical airport engines.) Figure 4-1. The simple substitution method for EDMS/AEDT data. Figure 4-2. The simple substitution method for experimentally measured data.

Sensitivity Analysis on Airport Emissions 25 Step 3—Sensitivity Analysis Two emission scenarios were compared for the hypothetical airport: • A baseline scenario using pre-existing data and • An updated scenario including experimental data. The variability of the experimental data was used to define limits for the updated scenario; thus one is 95% confident that the hypothetical airport emissions fall between the upper and lower confidence limits. The baseline scenario largely fell within these confidence limits. Even though some observed changes were big (e.g., ~194% for hydrocarbon emissions), the differences were not statistically significant. A sensitivity analysis using EDMS/AEDT was performed to determine the potential effect of replacing existing emissions indices in the EDMS/AEDT database with those derived during the ACRP Project 02-54 sampling field campaign on computed emissions. For this analysis, the research team used the latest version of FAA’s EDMS (Version 5.1.4.1). The FAA released the new AEDT model in May 2015, with the current release of AEDT2b (Service Pack 2) released December 22, 2015. As detailed on FAA’s website, AEDT was addressing multiple bug fixes, including known issues with user-defined aircraft and emission reports. Given current and pending updates to AEDT, ACRP Project 02-54 focused on using the EDMS model for its com- parison, because the results computed would be similar to those of AEDT. To add new aircraft to EDMS, the User-Created Aircraft option was invoked in EDMS. This application allows a practitioner to create a user-defined aircraft; assign it a flight profile; des- ignate other operational characteristics that have a bearing on emissions calculation; and, most significantly, input measured carbon monoxide (CO), hydrogen carbons (HC), nitrogen oxides (NOx), and smoke number (SN) emissions indices that are divergent from available EDMS/ AEDT information. Figure 4-3 is a screenshot of a user-created menu of available options for this function within EDMS/AEDT. Similarly, to create a user-defined aircraft in AEDT, the user must copy data from an aircraft that already exists in the AEDT database and modify the emission indices under engine emission coefficients (see Figure 4-4) with the new data. Detailed instructions on how to create user-defined aircraft are presented in the AEDT 2b User Guide, December 2015 (Koopmann et al. 2015). The sensitivity analysis was performed at the hypothetical GA airport at an airport fleet level (i.e., the full complement of GA aircraft operating at an airport with the full level of operations assigned). Hypothecal Fleet Sampled Engine Model EDMS/AEDT Match Comments Cate gory Aircra Make Aircra Model Engine Family Aircra Make Aircra Model Engine Model SEP CESSNA 172 O 320 O 320 Cessna 172 0-320 Exact match with Hypothecal Fleet and sampled aircra/engine. CESSNA 182 O 470 O 470 Cessna 182 IO-360-B No O 470 in EDMS; chose IO 360 because (only opon for Cessna 182) similar horsepower (hp). CESSNA 150 O 200 O 200 Cessna 150 O-200 Exact match with Hypothecal Fleet and sampled aircra/engine. PIPER PA 28 O 320 O 320 Piper PA-28 O-320 Exact match with Hypothecal Fleet and sampled aircra/engine. CESSNA 172 O 300 Cessna 172 O-320 No O 300 in EDMS; chose O 320 because similar hp. Table 4-2. Example mapping of five hypothetical fleet aircraft to their equivalent engines in sampled and EDMS/AEDT datasets. (Full mapping is available in Appendix L)

26 Exhaust Emissions from In-Use General Aviation Aircraft Figure 4-3. EDMS user-created aircraft modeling options. For each aircraft replaced with a surrogate with derived emissions indices resulting from the ACRP Project 02-54 sampling field campaign, emissions were computed under three scenarios as follow: 1. Baseline Scenario: using EDMS/AEDT aircraft with its default/existing information, 2. Updated Scenario (User-defined Averages): populating EDMS/AEDT with averages of the user-defined alternatives representing the refined data from the sampled field campaigns, and 3. Updated Scenario (User-defined Upper Limits): populating EDMS/AEDT with upper limits of the user-defined alternatives representing refined data from the sampled field campaigns. For this analysis, the upper limit consisted of the 95% confidence interval estimate, which reflects a significance level of 0.05. These emission inventories were produced and compared to assess the aggregate change in emissions on an aircraft/engine-specific level (i.e., aircraft operational modes). For example, it would be possible to disaggregate the results and attribute the change to a specific aircraft mode whose emissions indices for that specific mode substantially changed as a result of the research effort. For consistency and to focus on the effects of the new emission indices, all other standard EDMS/AEDT input data (e.g., operational times-in-mode for taxi/idle, take-off, etc.) were used. The default approach and climb-out times were based on standard ICAO/EPA data up to an altitude of 3,000 feet. Default taxi-in and out times of 7 and 19 minutes were also used. To estimate the number of operations per aircraft in the Hypothetical Fleet, equal use through- out the year was assumed for each aircraft, because operational data is not publicly available. The total annual operations from 20 GA airports in the United States were averaged using data from FAA’s Operations Network, giving an average of 97,192 operations (i.e., 48,596 landing/take-off (LTOs)) per year per airport. The number of operations of each aircraft was estimated based on its occurrence over the total averaged operations (Table 4-1) within the Hypothetical Fleet.

Sensitivity Analysis on Airport Emissions 27 Figure 4-4. AEDT user-defined aircraft modeling options. Baseline Scenario Under the baseline scenario, the current EDMS/AEDT databases of GA aircraft engine emission indices were used to populate the scenario using the Hypothetical Fleet. EDMS was populated either with an aircraft/engine with an exact match to a sampled aircraft/engine or a surrogate based on similar aircraft type/weight and engine operational characteristics (i.e., emission coef- ficients and horsepower) from the EDMS/AEDT database. Table 4-2 (with additional values in Table L-1) lists the aircraft/engines sampled during the ACRP Project 02-54 field campaign and the corresponding surrogates used in EDMS/AEDT. Explanations for why each aircraft and engine assignment were chosen in EDMS are also listed. Substitutions were made by choosing an aircraft/ engine with more conservative (i.e., higher) engine coefficients and/or higher aircraft weight. Using surrogate GA aircraft/engines emission indices for aircraft not in EDMS databases has become the standard operating procedure (SOP) when using the model. Therefore, among the objectives of the ACRP Project 02-54 research was to show the need to expand this limited database of GA aircraft emission indices and provide model users with a greater range of aircraft/engine choices.

28 Exhaust Emissions from In-Use General Aviation Aircraft Updated Scenarios The updated scenarios examined consisted of (1) using averages of samples collected and (2) using the upper limits (within a 95% confidence interval) of samples collected: • Average Scenario: Averages of each aircraft/engine sampled indices within each mode/thrust setting were entered into EDMS for comparison with the baseline scenario and • Upper Limit Scenario: Upper limits were calculated as the 95% confidence interval and entered into EDMS for comparison with the baseline scenario. Some aircraft did not have sufficient data/number of samples for upper limits to be computed. Under the updated scenarios, new GA aircraft engine emission indices derived from the ACRP Project 02-54 sampled field campaigns were added to the EDMS database using the model’s User-Created Aircraft option (see Figure 4-3). Total PM mass emissions (tPMm) were the only measure of PM considered here. Additional parameters such as, but not limited to, the number of engines, aircraft/engine category, and flight profile were entered. The times in mode for each aircraft were left as EDMS default times. If a mode was not measured in the updated scenarios, the default values for that mode were kept. This process was repeated for each of the aircraft sampled in the field campaigns. Results of the Sensitivity Analysis Table 4-3 summarizes the emission inventory results (in short tons per year) when comparing the baseline scenario (EDMS default values) to the Updated Average and the Updated Upper Limit Scenarios, respectively. Only aircraft and ground support equipment (GSE) are reported, because auxiliary power units (APUs) are not present in the Hypothetical Fleet selected. Measure Scenario CO2 CO THC NMHC VOC TOG NOx SOx PM 10 PM 2.5 Fuel Consum pon Aircra Baseline 2,909 1,048 21 19 18 21 2 ~1 3 3 922 Updated Average 3,198 989 63 70 70 71 3 ~1 ~1 ~1 1,014 Updated Upper Limit 3,493 1,780 244 279 277 280 18 ~1 2 2 1,107 GSE Baseline N/A ~1 N/A 0.1 0.1 0.1 0.3 <0.1 <0.1 <0.1 N/A Updated Average N/A ~1 N/A <0.1 <0.1 <0.1 0.1 <0.1 <0.1 <0.1 N/A Upper Limit N/A ~1 N/A <0.1 <0.1 0.1 0.2 <0.1 <0.1 <0.1 N/A Totals Baseline 2,909 1,049 21 19 18 21 2 ~1 3 3 922 Updated Average 3,198 990 63 70 70 71 3 ~1 1 ~1 1,014 Updated Upper Limit 3,493 1,781 244 279 277 280 18 ~1 2 2 1,107 % Difference (average) 10 6 194 275 288 238 64 10 66 66 10 % Difference (upper limit) 20 70 1,046 1,391 1,447 1,235 861 20 31 31 20 Table 4-3. EDMS/AEDT results comparison for baseline, average and upper limit scenarios (short tons per year).

Sensitivity Analysis on Airport Emissions 29 When all aircraft results are averaged together, results comparing measured emission indices to the EDMS default values indicate that overall increases in emissions ranging from 10 to 288% are revealed for carbon dioxide (CO2), total hydrocarbons (THC), non-methane hydrocarbons (NMHC), VOC, total organic carbon (TOG), nitrogen oxides (NOx), sulfur oxides (SOx), and fuel consumption. By comparison, CO is shown to decrease by approximately 6%, and PM by 66%. PM10 and PM2.5 emissions are also shown, where PM10 is a measure of particulate matter mass that is smaller than 10 µm in diameter. The results of the sensitivity analysis include more than the four emissions species input: EDMS has partitioned certain species into subcategories. For example, THC (or HC) has been used to calculate the values for NMHC, VOC, and TOG. This partitioning uses a set of factors built into EDMS that are chosen based on EPA guidance. Factors are different for turbines and for pistons. In a similar way to HC, PM10 emissions are used to determine PM2.5. Investigating these factors for GA is one area for future research. In fact, as a part of this project, data was collected that would allow comparison of many different emission ratios (e.g., VOC/THC) and would help verify or redefine these partitioning factors. PM size measurements were also col- lected and show the partitioning of PM sizes. For example, piston engine PM sizes are typically smaller than 20 nm (0.02 µm), more than 100 times smaller than the cutoff for PM2.5. Figure 4-5 shows the results from this sensitivity analysis for four main emissions compounds. The solid bars are the baseline or updated scenario averages, while 95% confidence limits are shown with a thin capped line. The upper limits are significantly higher than the average values, because of the large variability of emissions for piston engines. In all cases except CO, the lower limits reach 0 (emissions cannot be negative). Figure 4-5 also demonstrates that only extreme changes in a GA airport’s emissions will be statistically significant (true with 95% confidence). In fact, the large changes in the updated scenario are not statistically significant except for PM10. The change in PM10 emissions is not surprising: the EDMS/AEDT data for piston engines all have the exact same PM emission factors, a sign that these are default values. Overall, piston engine emissions variability turns out to be much more important than the updates in individual emission factors. The effect of emissions variability is important in a regulatory framework, because when the upper limit emission coefficients are used, all indices increase significantly, with the exception 2000 1500 1000 500 0 C O [M g/ yr ] Baseline Updated 250 200 150 100 50 0 H C [M g/ yr ] 15 10 5 0 N O x [M g/yr] Baseline Updated 4 3 2 1 0 P M 10 [M g/yr] Figure 4-5. Results of the hypothetical airport sensitivity analysis showing baseline and updated results (solid bars), along with 95% upper confidence limits on the updated results. All emissions are given in mega grams per year (Mg/yr).