Summary

The U.S. Department of Energy (DOE) issues “standards regulations” for energy conservation pursuant to the Energy Policy and Conservation Act of 1975 (EPCA), as amended, and other authorities. These standards regulations apply to certain consumer products and commercial and industrial equipment. These can include air conditioning and heating systems, washing machines, and commercial refrigeration, among numerous other examples. DOE issues standards regulations by rulemaking and includes quantitative maximum water and energy use or minimum energy conservation standards. There are currently standards regulations for more than 70 product classes (i.e., a specific type of consumer product or commercial or industrial equipment).

This study of the Committee on Review of Methods for Setting Building and Equipment Performance Standards is a review of the assumptions, models, and methodologies (“analytical methods”) that DOE uses in setting the quantitative portion of the standards regulations following the Office of Management and Budget’s (OMB’s) guidance¹ on the use of scientific information. The EPCA includes a list of statutory criteria DOE must consider in setting these standards. In conformance with these statutory criteria, DOE has evolved a set of analytical methods it currently uses when setting a standard for a specific product class.

The analyses under review construct cost-efficiency relationships for products in a class to arrive at quantitative trial standard levels (TSLs) for the product class under consideration. The analyses also consider downstream effects of different quantitative standards on consumer life-cycle cost, manufacturer impact, employment impact, and impact on small entities. The committee’s task was to consider the methods employed in each analysis.

DOE provided the committee with the full suite of analyses for standards regulations for each of the following three product classes in the Appliance and Equipment Standards Program: (1) residential dishwashers, (2) commercial refrigeration equipment, and (3) residential furnaces. DOE had developed the analyses for these product classes to support specific rulemaking efforts (i.e., the administrative process of setting standards regulations). The three rulemakings provide examples of product categories in wide deployment. The three product categories are diverse in the type of energy service delivered, the novel issues they instantiated, and the consumer behaviors of relevance to each. Nonetheless, the committee found commonalities among them, and many of the committee’s findings and recommendations proved robust across all three illustrative product classes. DOE also discussed other standards from its Appliance and Equipment Standards Program—there are more than 70—in response to the committee’s queries.

___________________

¹ Office of Management and Budget, 2005, “Final Information Quality Bulletin for Peer Review,” Federal Register 70: 2664, January 14.

DOE also briefed the committee on its current practices for conducting and sequencing the analyses. DOE presenters explained the dependencies among the analyses and where the output of one becomes the input of another (see Figure S.1). The sections below provide a summary of the committee’s observations on the analyses that concern the hardware (i.e., the product or equipment itself) and the impact on producers and consumers.

POLICY CONTEXT

DOE establishes its standards regulations in the context of statutory authority and executive orders. EPCA, as amended, authorizes DOE to issue standards to “achieve the maximum improvement in energy [or water] efficiency . . . which the Secretary determines is technologically feasible and economically justified,” which involves evaluating “whether the benefits of the standard exceed its burdens.” In addition, the Administrative Procedure Act, the Regulatory Flexibility Act, and other laws concerning administrative process guide DOE’s issuance of regulations. The executive branch exercises its discretion as well, and the Regulatory Impact Analysis (RIA) framework, created by presidential executive order and subject also to OMB Circular A-4, provides a structure for analyzing and developing new and amended standards and for reviewing existing standards. To develop standards consistent with its statutory and executive authority, DOE has designed the series of analyses depicted in Figure S.1. While DOE does prepare RIAs for its energy efficiency standards, it does not integrate them into the standard-setting framework but conducts them as a separate analysis after key decisions have been made. The RIA framework is used across the federal government, including at agencies with related missions (such as the EPA, which also seeks to address environmental externalities), so its use would improve transparency and effectiveness across federal policy. Furthermore, because the RIA framework is consistent with DOE’s statutory mandate to ensure its standards are technologically feasible and economically justified, organizing its analysis according to the RIA framework—for example, (1) identifying material failure of

private markets or other compelling public need, (2) analyzing the benefits and costs of alternative approaches to addressing that need, and (3) evaluating the distribution of those benefits and costs—could improve the consumer impacts and societal outcomes of DOE’s standards.

To better understand the social impacts of its standards, DOE should organize its analysis following the regulatory analysis framework laid out in OMB Circular A-4. For example, rather than presenting the RIA as the last stage in the process, the analyses shown in Figure S.1 could be technical appendices to the RIA, which would integrate this information in presenting estimates of the net benefits and distributional impacts of the proposed efficiency standard and reasonable alternatives, referencing information from relevant appendices as it describes a baseline scenario and cost and benefit models. (Recommendation 2-1)

DOE should pay greater attention to the justification for the standards, as required by executive orders and the EPCA requirement that standards be economically justified. DOE should attempt to find significant failures of private markets or irrational behavior by consumers in the no-standards case and should consider such a finding as being necessary to conclude that standards are economically justified. (Recommendation 2-2)

DOE’s regulatory analyses that support tighter standards are often premised upon an energy efficiency gap—that is, DOE’s engineering analysis demonstrates that readily available technologies save far more in energy costs than they require in upfront costs. However, the RIA does relatively little to interpret this fact. Not all interpretations of the energy efficiency gap may imply a market failure. Other explanations include biases in the cost estimates, a failure to properly account for performance trade-offs, a failure to account for irreversibility and option value, a failure to account for borrowing constraints, and mismeasurement of actual energy efficiency.

DOE should place greater emphasis on providing an argument for the plausibility and magnitude of any market failure related to the energy efficiency gap in its analyses. For some commercial goods in particular, there should be a presumption that the market actors behave rationally, unless DOE can provide evidence or argument to the contrary. (Recommendation 4-13)

DOE should give greater attention to a broader set of potential market failures on the supply side, including not just how standards might reduce the number of competing firms, but also how they might impact price discrimination, technological diffusion, and collusion. (Recommendation 4-14)

SCREENING AND ENGINEERING ANALYSES

The objective of the Screening Analysis and Engineering Analysis is to identify TSLs and develop cost-efficiency relationships by estimating the manufacturer’s costs of achieving increased efficiency levels and determining the maximum technologically feasible efficiency level. Overall, the committee found that the approach DOE used was sound, but it could be improved.

The cost-efficiency relationship developed and used by DOE currently assumes that the only two determinants to be considered in appliance standards are cost and efficiency. However, changes in wholesale markets for electricity now necessitate that assessments of standards include an evaluation of demand response readiness. It has also become important to include a thorough evaluation of the impact that demand-side management (DSM) programs and activities can have on leveraging a new or revised standard’s effectiveness.

DOE should consider demand response readiness as a factor in cost-efficiency calculations. This necessitates the inclusion of power system benefits not currently considered. (Recommendation 3-6)

These additional elements necessitate that the engineering analysis include a more detailed understanding of the projected energy consumption patterns and amounts for appliances. Elements for DOE to consider include ranges of costs, patterns of consumption by seasonal time of use (day, week, and month) and (heterogeneous) pattern of consumption by user, diversity factors, energy peak demand, and variance regarding environmental factors. DOE often uses engineering estimates to develop these data. The energy consumption patterns and amounts that are projected for an appliance will need to include results by seasonal time-of-use (day, week, and month); these projections will be important for the RIA as well. The use of actual test data, including experience from previously implemented standards in place of computer simulation or to validate computer simulations, will improve the analytical outcomes.

DOE should expand the cost analysis segment of the engineering analysis to include ranges of costs, patterns of consumption, diversity factors, energy peak demand, and variance regarding environmental factors. (Recommendation 3-5)

DOE screens out options during its analysis supporting the Notice of Proposed Rulemaking that are technologically infeasible or impractical. It also screens out options that will adversely impact society or the product utility and availability. The current screening analysis is conservative and may force the exclusion of some otherwise desirable technology options. The most effective appliance efficiency standards would encourage the development and eventual adoption of innovative technology. Other agencies use alternative methods in their own technology evaluations for identifying a technology’s readiness, such as the technological readiness level (TRL) taxonomy developed originally by the National Aeronautics and Space Administration (NASA).

DOE should consider technologies that are at early, pre-competitive technological readiness levels and have promise for use in consumer products and commercial/industrial equipment as part of product population analyzed, even if it seems plausible that they will be screened out in later stages of the analysis, such as in the screening analysis made during the Notice of Proposed Rulemaking. DOE should continue to use the tools at its disposal, such as reconsidering a previously excluded technology, to avoid prematurely screening-out innovative technologies. (Recommendation 3-1)

DOE often uses engineering analysis as a substitute for monitoring actual performance of new or improved appliances. Field tests of a cohort of buildings to test alternative technologies would be one way to ensure optimal evaluation.

DOE uses point estimates of efficiency levels, but there is an opportunity to characterize the uncertainty of the efficiency ratings as experience in deployment for a given appliance. Performance can depend on characteristics of the environment within which the appliance or equipment is installed and on how homeowners or commercial businesses use the appliance. Thus, rather than providing a “point” estimate of efficiency, DOE could provide a range that reflects the variability in energy consumption under different uses. In the future, sensor data may provide great insight into use patterns that allow DOE to understand impacts on low-frequency users of the technology and therefore improve distributional impact data.

The ongoing explosion in the use of sensors, computational ability, and communications creates an ever-increasing library of consumption data at the appliance, building, and power system level. A large percentage of U.S. residential electricity consumers now have smart meters that can record and communicate consumption data. Many new appliances are web-connected and incorporate added functionality to their products that permit the manufacturer to monitor and manage consumption. This

data influx is an opportunity. However, some of these data may generate privacy concerns that will need to be addressed.

WELFARE ANALYSIS OF CONSUMERS AND PRODUCERS

The economic analysis translates changes in the costs and attributes of appliances and equipment into changes in seller profit, consumer well-being, and environmental quality. The committee’s charge focused on the methods employed in this portion of DOE’s analysis.

The committee’s report notes various ways that DOE’s analysis deviates from an ideal economic analysis and offers numerous concrete steps for incremental improvement over a range of topics. But, even more, the report concludes that there is a great deal of uncertainty that cannot be resolved because economic models are ambiguous on key points, because research on other key points is unsettled, and because DOE lacks necessary data. Thus, the committee’s broadest recommendations pertain to (1) quantifying and presenting uncertainty and (2) collecting additional data.

Regarding uncertainty, DOE’s economic analysis treats uncertainty around key parameters inconsistently and in segmented ways. The committee advocates a more systematic and integrated representation of uncertainty that propagates uncertainty throughout the analysis. It also focuses on parameters most likely to be pivotal to decisions and makes sharper distinctions between variability and uncertainty. Finally, the committee provides a recommendation for DOE to present a complete picture of the consequences of alternative standards to the final decision-makers.

In order to evaluate the economic costs and benefits of a standard, DOE should present the distribution of costs and benefits estimated in its models when (1) uncertain parameters are represented by probability distributions and (2) parameters that vary across geographic and other relevant dimensions are disaggregated. The uncertainty or variability the parameters represent should be compounded or propagated—properly accounting for any correlations—throughout the calculation. This methodology is necessary for the markup analysis and manufacturer impact analysis, the shipments analysis, and all components of the life-cycle cost analysis. Where multiple sources of uncertainty must be combined for the final benefits result, as with net benefits depending on both the shipments analysis and the appliance unit cost and performance, the subcomponents should be reported as well. (Recommendation 4-15)

Regarding additional data, the challenge is in providing detailed, forward-looking cost-benefit analysis in markets that deviate in significant ways from the benchmark model of rational, competitive markets, and about which there is frequently a lack of hard data. True improvements are likely to come from gathering and using new data, rather than from grafting more sophisticated economic models into the core of the analysis.

Several new data types could prove invaluable, including better information on product prices and markups (to ground models of seller behavior); more information about consumer preferences (to quantify attribute-energy trade-offs) and usage (to better model variability); as well as measures of in situ product performance (to more accurately measure energy consumption of products as deployed and used). Perhaps most important, given the extent of uncertainty in how these markets operate, how new generations of appliances operate and how technological change will impact both supply and demand, is a greater store of ex post analysis (i.e., after the standards have had time to take effect) that validates assumptions made in prior standards and evaluates the implications of prior forecasts’ inaccuracies and mistakes.

DOE should obtain better data for improving the economic analyses of appliance and equipment performance standards. Empirical data are needed on markups, consumer choices in appliance markets, and in situ performance. Some of this information can come from

relatively simple changes to current surveys and studies, including engineering analyses of the Appliance and Equipment Standards Program and the Residential Energy Consumption Survey. (Recommendation 4-16)

Conducting an evaluation, ex post, of the performance of appliances and equipment can permit evaluation of the validity of the ex ante assumptions relied upon in setting standards in the first instance and will allow learning and iterative improvement of the analysis. This can be significant when economic models require unverifiable assumptions, critical inputs are highly uncertain, or relevant data are unavailable.

DOE should commit to collecting data necessary to conduct more rigorous ex post analysis of the effects of standards on consumers, producers, energy consumption, and environmental impacts. (Recommendation 2-3) Data are in particular needed on prices and markups (Recommendation 4-1), consumer choices (Recommendation 4-3), and in situ performance (Recommendation 4-8).

Ex post analyses can validate assumptions made in prior standards and evaluate the implications of prior forecasts’ inaccuracies and mistakes. DOE should use such ex post analyses routinely to improve forward-looking standards iteratively. (Recommendation 4-17)

An integral part of the welfare analysis for energy efficiency programs is the quantification of emissions reductions. While DOE does well to monetize the changes in emissions quantities using the most recent outside studies and government reports on the economic and health damages associated with emissions, it also appears not to monetize some pollutants, nor does it represent or communicate all of the uncertainty and variability in these economic numbers. The problem is complex as emissions reductions critically depend on other public policies. For example, the extent of penetration of renewable generation in the electric grid, whether pollution caps are binding for emissions subject to cap-and-trade programs or the implementation of demand-side management or demand response programs. However, because of the importance of emissions reductions to evaluating prospective standards, the committee recommends that DOE expand its valuations of emissions reductions.

DOE should monetize all emissions impacts for which meaningful estimates of social impacts are available, considering policy interactions when those interactions are deemed significant. (Recommendation 4-12)

CONTEXT OF ENERGY CONSERVATION STANDARDS

DOE considers alternatives to any new or revised appliance and equipment standards when drafting the RIA that accompanies the rulemakings by which it issues new or revised standards. These alternatives are not standards regulations but rather involve other types of government interventions related to, for example, taxes or rebates that affect consumer choice; voluntary targets perhaps combined with information campaigns; government purchases; and the no-action alternative. The RIA presents an opportunity to estimate the effectiveness of other means of achieving the goals of the Appliance and Equipment Standards Program. For example, there are a number of attributes sought by the consumer beyond low energy costs, including maintaining comfort, safety, convenience, environmental impact, security, control, appearance, and high-tech attributes.

The RIA process for the three rulemakings assessed by the committee evaluated DSM elements related to consumer rebates and consumer and manufacturer tax credits as alternatives to standards. Analyzing these elements as an integrated program, rather than individually, would more adequately

assesses the DSM program’s effectiveness as a complementary effort to the Appliance and Equipment Standards Program.

A further consideration is the impact of a standards program on the load shape (purchase pattern of demand). To understand such impact, an estimate can be made through a capacity expansion and production costing model to understand the impact on the need for capacity and the cost of electricity.

FUTURE STANDARDS REGULATIONS

There are many trends in technology, regulation, and business models that will influence the structure and effectiveness of appliance efficiency standards, including information and communications technology (ICT) enabling the Internet of Things (IoT); the evolution of the integrated grid; the move toward electrification; and regulatory changes involving electricity markets that can transform the role of appliance standards in achieving energy efficiency. This transformation is an opportunity for DOE to develop a standards program that encompasses the entire scope of energy utilization—appliances, their installation, controls, and the built environment.

The IoT connects smart meters, smart homes, and smart buildings, providing increased visibility into real-time electricity demand. This connection may directly impact efficiency standards. Utilities, consumers, and electricity suppliers can use the information to match generation resources accordingly. Vice versa, smart meters and mobile apps will provide consumers with real-time pricing of electricity. Consumers can match their use of electricity to reduce their cost, which reduces peak demand and levels out the consumption profile, providing both more efficient generation and use of electricity. These features can dramatically affect the impact of energy efficiency standards.

In the future, DOE may need to modify appliance standards’ applicability because of emerging integrated grids, regardless of how widely distributed or centralized they are. In the integrated grid concept of operations, appliances must be flexible enough to optimize the value of a combination of local generation and energy storage, along with energy efficiency and new uses of electricity integrated with central generation and storage. In this new integrated grid, energy efficiency includes both appliances and the buildings or systems that house them as distributed resources.

Appliance and equipment standards are ideally never a factor constraining or delaying technologies that add flexibility to electrical loads. Information must be available regarding estimates of energy savings at different times of day and seasons, and the value of those savings under different assumptions on future penetration of renewable energy.

There are many changes in the U.S. electricity marketplace that will affect DOE’s decisions when creating prudent standards. One change is “fuel switching,” or programs created to encourage consumers to electrify. For example, California enables municipalities to adopt building codes that prohibit the use of natural gas in buildings.

The IoT provides the framework that may allow DOE to shift its focus from the efficiency of an individual appliance in ideal conditions to a broader focus that includes installing and operating the appliance and its actual demand and pattern of usage governed by consumer behavior.

Preparing for the future will require collecting a broader array of data and information on overall energy consumption. Disaggregated data allows energy system planners, regulators, and policymakers to

best judge equitable policies, adjudicate tariffs for regulated entities, and plan critical community infrastructure.

The energy industry is increasingly exploring the use of analytics to mine value from customer data. These data may lead to the ability to modify methods for setting and evaluating appliance and equipment standards, resulting in a standards program that could look very different than it does today.

Restructured electricity markets seem inevitable, evidenced by the numerous states that have adopted some pieces of restructured markets. When setting or revising the appliance and equipment standards, DOE can ensure that standards do not slow or preclude the development of these markets. However, while such markets exist today, certain market barriers still exist. These barriers—including a lack of information, the absence of qualified installation contractors, and limited financing options—inhibit energy consumers from fully comprehending all cost-effective energy-efficient products and services.

CLOSING

The current Appliance and Equipment Standards Program includes a sequence of analyses good at finding a standards level that can be met by current technology. However, DOE can continue to meet its statutory obligations under EPCA, as amended, and improve the analytical methods applicable to its 70-plus categories of appliances and equipment subject to the energy and water conservation standards.

The lack of empirical data make improving the analysis a challenge, and changes to existing surveys could improve this, as would moving away from the use of point estimates. Ex-post analysis of the costs and benefits of the standards would both provide a check on the assumptions made when the rulemaking was promulgated and offer a guide on how to improve future standards. The extent of uncertainty in estimating costs and benefits of proposed standards underscores the value of analyzing the underlying market conditions that suggest standards regulation will be beneficial. To that end, framing the analyses using the RIA principles is a useful refinement. Finally, a more forward-leaning analysis might capture more of the potential for the leading edge of technology to enter the market and thereby increase the energy and water conservation possible.