Lawrence Berkeley National Laboratory

EXECUTIVE SUMMARY Expectations about technological change are central to the way policy details are set in a variety of complex public problem areas, including health, welfare, safety, and environmental protection. These details are established through a regulatory process designed to incorporate scientific, technical, economic, and other expertise while following principles of good governance such as transparency, analytical rigor, and the appropriate treatment of uncertainty. An important element of most U.S. federal regulatory processes is cost-benefit analysis, which is expected to employ “best-available” techniques that identify “changing future compliance costs that might result from technological innovation or anticipated behavioral changes (OMB 2011). Influential calls for the reform of regulatory analysis, however, locate the appropriate modeling of technological change “at the frontier of economic research.” (Harrington, Heinzerling et al. 2009). There are multiple reasons why it is difficult to appropriately model technical change in regulatory analyses. First, even scholarly expectations of the influence of regulation on the rate and direction of technical change are still strongly influenced by two opposing contentions. One contention is that innovation can be induced to both public and private benefit by policy that is well-designed, particularly with respect to stringency, technological neutrality, and uncertainty (see, e.g., Porter and Van der Linde 1995, Ruttan 1997, Taylor, Rubin et al. 2005, Kemp and Pontoglio 2011). The other contention is that policy interventions can be expected to hamper innovation due to the influence of asymmetric information between the public and private sectors (see, e.g., Stewart 1981). Second, although the Office of Information and Regulatory Affairs (OIRA) in the Executive Office of Management and Budget (OMB) – the agency tasked with providing oversight of the quality of U.S. regulatory analyses – has called for the use of best available techniques to account for innovation in regulatory analyses, it has not suggested that specific techniques be employed (which it has done in other areas of analytical methodology). Third, there is no easily identifiable list of the technological change-dependent model assumptions, inputs, and calculations used to generate estimates of the costs and benefits associated with any given regulation. Fourth, regulated industries do not provide detailed guides to their approaches to innovation, which vary by industry (e.g., the pharmaceutical industry does not approach innovation the same way that the appliance industry does, although both are subject to regulation). Retrospective reviews of the accuracy of regulatory cost estimates point to the possibility that there may be real value involved in trying to improve this situation. Over the past forty years, about three-quarters of the 60+ U.S. regulatory cost estimates that have been retrospectively reviewed have proven to be significantly inaccurate, where accuracy is defined as ex post costs falling outside the range of +/- 25% of ex ante estimates, in keeping with a standard established in Harrington, Morgenstern et al. (2000). As discussed in Simpson (2011), the majority of these inaccurate regulatory cost projections are over-estimates of the costs of regulation, and researchers have not been able to reject the null hypothesis that this robust finding is evidence of systematic bias. The leading conjecture offered to explain this high level of cost-overestimation – and its implicit corollary with respect to leaving societal benefits “on the table” – is that rulemaking analyses are not currently able to accurately account for future technical change in industries subject to regulation (Harrington, Heinzerling et al. 2009). iii The calls in recent presidential Executive Orders and in recommendations by the Administrative Conference of the U.S. for periodic retrospective reviews of existing rules provide an opportunity to start to address the situation with some rigor. The purpose of retrospective review is to bring empirical evidence to bear on important topics that can inform the quality of future regulatory efforts. The retrospective review project we report on in this report represents an important first step in marshalling evidence to inform the expectations of technical change which are embedded in the rulemaking process of a prominent area of national regulation. It takes a layered approach to empirically considering regulation and innovation, starting with the tangible outcomes of technical change and other factors that can be predicted through regulatory analysis and observed in the marketplace, such as the price and quality of regulated products. It then explores some of the subtler ways that the regulated industry’s approach to innovation may not be reflected in the rulemaking process, to the potential detriment of analytical accuracy. In doing this, it identifies some potentially valuable areas for future research that can inform refinements to the rulemaking process as well as the broader scholarly debates about regulation and innovation. The area of regulation focused on in this project is minimum efficiency performance standards (MEPS). Although this regulatory area has been particularly active in recent years because of its relevance to U.S. climate change policy efforts, for the purpose of retrospective review it is best understood in the broader historical context of U.S. energy policy. Since at least the 1970s, it has been recognized that appliances and other energy-using products make significant contributions to the energy consumption of the residential building sector (currently estimated at 22% of U.S. energy use). The basic concept underlying MEPS is that it removes from the market certain models of covered appliances and other energy-using products that do not meet specified energy efficiency thresholds. The covered products studied in this project are room air conditioners (“room ACs”), refrigerator-freezers (“refrigerators”), dishwashers, clothes washers, and clothes dryers. These five products, which heavily saturate U.S. households, are the full set of large domestic appliances that were subject to MEPS regulatory analyses conducted by the U.S. Department of Energy (DOE) prior to 2012. The research advantages of focusing on these appliances include limiting selection bias, ensuring the likelihood of sufficient data for retrospective review, and narrowing the focus to products that serve similar markets albeit with varying regulatory histories. Note that we ground our retrospective analyses of these products in a broader market and policy context that is informed by earlier retrospective reviews of MEPS that drew on lowerresolution data. The large amount of data that informs this project can be characterized as either ex ante (expectation) data or ex post (observation data). The public rulemaking record provided a considerable amount of useful material to inform ex ante data, both in the form of technical support documents for specific rulemakings and petitions to reconsider final rules. Meanwhile, the report drew upon several sources of ex post data, including: (1) extensive 2003-2011 U.S. point-of-sale data on appliance models which we helped match to model energy use data (for all but clothes dryers) in order to allow us to construct a monthly panel of model-specific prices, quality characteristics, and market shares; (2) datasets we constructed from independent thirdparty appliance testing and surveys that speak to the quality and reliability of all five products; and (3) a dataset we constructed and coded that consists of the features identified in the product manuals of 1,109 clothes washer models sold in the U.S. in 2003-11 (these models represented iv 95% of the identifiable models in our point-of-sale data, which account for 29% of U.S. units sold over that period). The first set of analyses we report on demonstrated that purchased products in the U.S. surprised regulatory expectations regarding cost and quality. Using the definition of significant misestimation mentioned above, ex ante regulatory cost projections significantly over-estimated the majority of model price observations across all efficiency levels. By product, significantly overestimated observations were: > 95% of room AC observations; 54-66% of refrigerator observations; 42-72% of dishwasher observations; and 50-81% of clothes washer observations (clothes dryers had to be modeled in a different way, but the results were very similar). In addition, we show that within-model (i.e., with fixed effects) average prices declined for the refrigerators and clothes washers that U.S. consumers bought over the 2003-11 period. Meanwhile, the monthly sales-weighted average energy use of these products was better than the standard for almost all the data points of our study period, and in the one case for which we have relevant data (clothes washers), we show that regulatory analyses underestimated how enthusiastically U.S. consumers would buy highly efficient products. Our results show that these better-than-expected price and efficiency outcomes did not occur to the general detriment of the availability of products with high quality performance attributes other than energy use, despite stakeholder concerns and the revealed concerns of the designers of the MEPS rulemaking process in 1996. Instead, in most cases the statistically significant changes that occurred in thirdparty quality variables assessed across MEPS events were quality improvements. Similarly, the rate of significant repairs after five years of product ownership declined across our study period, according to third-party surveys, which points to increased product quality over the period of time products have been regulated by MEPS. Although our analyses do not provide causal evidence that MEPS induced innovation to both public and private benefit, as realized in these positive indicators of post-regulatory product price, efficiency, and quality, our results align with this contention much more than they do the contention that regulation hinders innovation. They also suggest that the mechanism by which MEPS could induce innovation relates to product design. The significance of product design at an aggregate level is apparent in our results in the stronger effect of product design versus efficiency levels in differentiating the within-model product price declines mentioned above. At a more disaggregate level, however, our exploratory feature-level analyses provided important insights regarding two aspects of product design that may have relevance to the rulemaking process followed for MEPS and to scholarship on regulation-induced innovation. First, we identify a high degree of feature bundling in the clothes washer industry that may occur in other appliances as well, either for technical and/or strategic reasons. Second, we observe that product energy use is a “core” product performance attribute contributed to by many technical features that also contribute to a range of other non-regulated product performance attributes. We make the case that regulatory-imposed constraints on this core product performance attribute have the potential to both reduce the search costs involved in product design while also stimulating cascading problem-solving across technical issues related to non-regulated product performance attributes, with the potential to enhance quality. Both insights would appear to be promising avenues for future research to improve the MEPS rulemaking process, as preliminary analysis does not support the idea that they are well-reflected in the current analytical process.