Lawrence Berkeley National Laboratory

We  present  a  methodology  to  evaluate  the  accuracy  of  baseline  energy  predictions.  To  evaluate  the  predictions from a computer program, the program is provided with electric load data, and additional data such as outdoor air temperature, from a “training period” of at least several months duration, and used  to  predict  the  energy  use  as  a  function  of  time  during  the  subsequent  “prediction  period.”  The  predicted energy use is compared to the actual energy use, and errors are summarized with several metrics, including bias and Mean Absolute Percent Error. An important feature of this methodology is that it can be used to assess the predictive accuracy of a model even if the model itself is not provided to the evaluator,  so  that  proprietary  tools  can  be  evaluated  while  protecting  the  developer’s  intellectual  property. The methodology was applied to evaluate several standard statistical models using data from four hundred randomly selected commercial buildings in a large utility territory in Northern California; the result  is  a  statistical  distribution  of  errors  for  each  of  the  models.  We  also  demonstrate  how  the  methodology can be used to assess the uncertainty in baseline energy predictions for a portfolio of buildings,  which  is  an  issue  that  is  important  for  the  design  of  utility  programs  that  incentivize  energy  savings. The findings of this work can be used to (1) inform technology assessments for technologies that deliver  operational  and/or  behavioral  savings;  and  (2)  determine  the  expected  accuracy  of  statistical  models used for automated Measurement and Verification of energy savings.