Skip to main content
Open Access Publications from the University of California

Method for Testing and Classifying the Effect of the Modeler on Building Energy Simulation Results

  • Author(s): Berkeley, Pamela Marie
  • Advisor(s): Carey, Van P
  • et al.

Concerns about global resource management and environmental conservation have drawn attention to the large amounts of energy used by buildings and the resulting impact on the environment. Building Energy Simulation (BES) programs play a crucial role in efforts to reduce energy use by the built environment. However, BES has many areas where sources of uncertainty may enter into the process and propagate to the results. In order for BES to reach its full potential for aiding in energy reduction efforts, a better understanding of the uncertainty in simulation results is required. Much research has been done to uncover the nature of these sources of uncertainty and additional work has been done to explore how the sources of uncertainty interact and propagate to the final simulation program output. Despite the extensive work already conducted on the topic of BES uncertainty, very little research has been done to investigate the effect of the building energy modeler on simulation results. This dissertation research explores the role of the modeler in the uncertainty of BES results, and establishes a testing methodology and classification system for sources of modeler variability. It additionally makes specific recommendations for the mitigation of each class of modeler variability.

A study was conducted where 12 professional building energy modelers were provided with identical building plans and asked to create a model of the building, in the BES program eQUEST, in accordance with their typical modeling habits. The building chosen for the modeling task was a single story school administrative building with a vaulted lobby, and it was located in California Climate Zone 4, the climate zone local to the sample of modelers. Time to complete the modeling task was limited to 3 hours to impose time pressure on the participants to expose how modelers prioritize different modeling tasks. All participants submitted the input and output files of the simulation for further analysis. Demographic information on the modelers was collected to determine if modeling decisions were linked to modeler background.

Various forms of analyses were employed to explore the study data and to develop a classification method for modeler variability. A one-at-a-time factor analysis (OAT) applied modeler decisions to a best-practices baseline model to assess the effect of individual participant decisions on simulation results. Monte Carlo sampling was applied to the set of participant decisions to create 200 input files that were hybrids of randomly chosen participant decisions. The results of this Monte Carlo analysis yielded the effect of modeler decisions in combination with all other modeler decisions. Classification trees were applied to the Monte Carlo data to investigate the interaction between modeler decisions. Random forests were applied to the Monte Carlo data to more robustly assess interaction effects. In the OAT, classification tree, and random forest analyses, the decision of how to represent the HVAC systems consistently was the most significant, so classification trees and random forests were applied to the individual HVAC system decisions to determine interaction effects with these parameters. Multiple Correspondence Analysis (MCA) plots were generated to explore any potential correlations between modeler background and modeler decisions.

A combination of the results of the OAT analysis and the random forest analysis yielded 3 basic classes of variability introduced by modelers. A high OAT impact and low random forest impact indicated that modeler decisions in that category differed from the best practices model consistently but had little impact on energy results when combined with other modeler decisions. Low OAT impact and high random forest impact categories were where modelers varied widely in their decisions; the decisions had very little effect on the baseline model on their own but had a large impact on results when combined with other modeler decisions. The final class of modeler variability was characterized by high OAT impact and high random forest impact. In this category, participant decisions varied greatly from each other and from the baseline model setting, and had a large impact on energy results independently and in combination with other modeler decisions. The MCA plots showed little correlation between modeler decisions and modeler background.

Future work needs to be conducted to confirm the classification system described above. Tests can be conducted on larger samples of modelers, on different sizes and types of buildings, on modelers and buildings in different climate zones, and in a variation of testing procedures. Furthermore, modeler variability mitigation tactics can be applied to the simulation process for each of the classes of variability to assess whether a reduction in modeler variability results from the mitigation tactics.

Main Content
Current View