This paper describes the implementation of the Building Controls Virtual Test Bed (BCVTB). The BCVTB is a software environment that allows connecting different simulation programs to exchange data during the time integration, and that allows conducting hardware in the loop simulation. The software architecture is a modular design based on Ptolemy II, a software environment for design and analysis of heterogeneous systems. Ptolemy II provides a graphical model building environment, synchronizes the exchanged data and visualizes the system evolution during run-time. The BCVTB provides additions to Ptolemy II that allow the run-time coupling of different simulation programs for data exchange, including EnergyPlus, MATLAB, Simulink and the Modelica modeling and simulation environment Dymola. The additions also allow executing system commands, such as a script that executes a Radiance simulation. In this paper, the software architecture is presented and the mathematical model used to implement the co-simulation is discussed. The simulation program interface that the BCVTB provides is explained. The paper concludes by presenting applications in which different state of the art simulation programs are linked for run-time data exchange. This link allows the use of the simulation program that is best suited for the particular problem to model building heat transfer, HVAC system dynamics and control algorithms, and to compute a solution to the coupled problem using co-simulation.