Smart thermostats are an uncomplicated and widely available technology for reducing HVAC energy use in homes, but how well do they actually work? Most existing methods for evaluating smart thermostats rely on data collected from installations. For instance, EPA requires 12 months of data from 1,250 qualified installations before granting the ENERGY STAR® label. This requirement makes it impossible to evaluate thermostats that have limited or no installations. Data-driven methods are also unable to directly compare multiple thermostats in controlled experiments because setpoint time series cannot be collected for multiple thermostats operating in the same home during the same period. With recent advances in energy simulation, these shortcomings can be overcome. In this work, we leverage these advances to create a simulation-driven framework for evaluating smart thermostats and use it to evaluate both a simple thermostat as well as a generic idealized smart thermostat algorithm for 52 representative homes with various types of HVAC equipment, extending previous results in this area. We found that on average, 5-10% energy savings are possible for our sample of homes. Some homes are able to exceed 20% savings, but probably with some discomfort to the occupants.

Open data is fueling innovation across many fields. In the domain of building science, datasets that can be used to inform the development of operational applications - for example new control algorithms and performance analysis methods - are extremely difficult to come by. This article summarizes the development and content of the largest known public dataset of building system operations in faulted and fault free states. It covers the most common HVAC systems and configurations in commercial buildings, across a range of climates, fault types, and fault severities. The time series points that are contained in the dataset include measurements that are commonly encountered in existing buildings as well as some that are less typical. Simulation tools, experimental test facilities, and in-situ field operation were used to generate the data. To inform more data-hungry algorithms, most of the simulated data cover a year of operation for each fault-severity combination. The data set is a significant expansion of that first published by the lead authors in 2020.