Servers Powered by a 10kW In-Rack Proton Exchange Membrane Fuel Cell System

.


INTRODUCTION
Fuel cell technology is an attractive electrical power generation technology receiving a great deal of recent attention. Fuel cells directly convert fuel to electricity. The direct electrochemical conversion of fuel allows for high fuel-toelectric conversion efficiencies without pollutant emissions. As reliable and environmentally friendly power sources, fuel cells can potentially play a very important role in data centers. A simple way of utilizing fuel cells as data center power sources is to connect them in grid parallel or backup generators. For example, eBay equipped its Utah data center with fuel cells from Bloom Energy [1] in a grid parallel configuration. The data center industry has experimented with centralized fuel cells through simulation and pilot installations. Such studies and demonstrations are mainly focused on: 1) installing high temperature fuel cells (several MW capacity) to power an entire data center [2][3][4], 2) advancing combined heat and power technology in the data center for better efficiency [5,6], and 3) performing economic and energy efficiency assessments [6,7]. Manno [5] simulated a cogeneration system based on a natural gas membrane steam reformer producing a pure hydrogen flow for electric power generation in a PEMFC. The study demonstrated that heat is recovered from both the reforming unit and the fuel cell in order to supply the needs of the data center. The possibility of further improving data centers' energy efficiency adopting DC-powered data center equipment is also discussed. Qu [6] reported a comprehensive performance assessment for a combined cooling, heating and power (CCHP) system with fuel cell in a data center. Data analysis and simulation results demonstrated great advantages of CCHP systems over conventional systems in the data center with regard to energy, environment and economic performance. Hagstotz [7] described the use of a molten carbonate fuel cell for data centers and telecommunication installations supplying cooling and electricity. Few of these published studies pertain to utilizing a midsize PEMFC system within the server rack that directly generates power, as is accomplished in this effort.
A key concern of using fuel cells to power servers directly is the load following capability of the fuel cells. Fuel cells themselves can usually respond sufficiently fast to changes in Proceedings of the ASME 2014 12th International Conference on Fuel Cell Science, Engineering and Technology FUELCELL2014 June 30-July 2, 2014, Boston, Massachusetts, USA 2 Copyright © 2014 by ASME load due to their rapid electrochemical reaction rates [8]. Processes inside the fuel cell such as electrochemical reactions and charge transfer processes typically occur over time periods on the order of milliseconds [9]. The main issue in fuel cell system load following is the relatively slow response of the fuel processing and fuel/air delivery subsystems. Since the electrochemistry directly produces the electrical work output, a fuel cell system should be able to achieve rapid load following capability on the same order as that offered by the electrochemistry. Load following problems occur when the response of the fuel cell system cannot safely meet both the external system power demand and the balance of plant power demand. The limitations could result from conservative control techniques or from inherently slow response of subsystem components, such as flow or chemical reaction delays associated with fuel/air processing equipment [9]. In the case of slow subsystem response the fuel cell performance depends upon the performance of the subsystems.
Different types of fuel cell systems exhibit different load following capabilities. The Solid Oxide Fuel Cell (SOFC) and PEMFC system response is fundamentally limited by the performance of the fuel preprocessor and the amount of hydrogen present in the anode compartment. During transients it is essential that sufficient hydrogen be maintained in the fuel cell to sustain the fuel cell voltage and avoid damage caused by hydrogen starvation. Fuel starvation in the anode compartment can occur if the fuel is consumed by the electrochemical reactions faster than it can be supplied by the fuel delivery system. Hydrogen is sustained in the anode compartment by controlling the fuel flow rate in proportion to consumed hydrogen and a desired utilization [10]. Other operating conditions such as feed gas humidity, operation temperature, feed gas stoichiometry, air pressure, fuel cell size and gas flow patterns were also found to affect the dynamic response capabilities of PEMFC [11,12].
In datacenters, the operation of servers generates large power changes in a relatively short period of time and the power sources must be able to follow the transient. Due to lack of studies, questions such as dynamic responses of the fuel cell/battery hybrid system and system integration and control are impeding the implementation of the fuel cell systems to datacenters. To further integrate fuel cell systems to data centers, transient analysis of the fuel cell system and battery components needs to be carried out combined with data center load changes.
In this study, we propose a direct generation method that places fuel cells at the rack level inches from servers. From the perspective of computer architecture, Distributed Fuel Cell limits the failure domain to a few dozen servers. Modern software technologies can tolerate such failures through replication and load balancing. As a result of our method, equipment such as power distribution units, high voltage transformers, expensive switchgear, and AC-DC power supplies in servers from data centers could be eliminated. To explore the feasibility and benefits of direct power generation in the server racks and the electrical properties of fuel cells, we demonstrated the use of a 10kW PEMFC stack and system as the distributed power source to power a server rack and eliminate the power distribution system in the data center. In this paper, the performance and the dynamic load following capabilities of the PEMFC system were evaluated and characterized.

EXPERIMENTAL CONFIGURATION
To analyze the steady-state and transient performance of fuel cell systems, a hybrid fuel cell-battery system was designed, installed and tested. In this paper, the PEMFC system used was a Hydrogenics HyPM TM 10kW Rack. The battery component was an APC Smart-UPS TM VT TM 15kVA UPS system. The hybrid system is designed to produce AC power output and the system and the schematic for the configuration tested are shown in Figure 1. The 12kW in-rack PEMFC system contained a fuel cell stack (rated at 12kW, 48VDC), a DC/DC converter (+/-192VDC), a system controller, and the Balance of the Plant. In the PEMFC system, ultra high purity H 2 was fed to the anode and ambient air was fed to the cathode. The supply of fuel and air were controlled by respective valves and blowers via the fuel cell controller. To reject the waste heat from the fuel cell power production, an internal water-based cooling circuit took the heat to a heat exchanger, where the heat was transferred to an external cooling circuit and rejected by a cooling subsystem external to the fuel cell rack using tap water and a radiator. Anode and cathode exhausts from the fuel cell stack were removed via a blower (400VDC, powered by the UPS during startup, and by the DC/DC output after the fuel cell was started) in the ventilation system which provides continuous ventilation and underpressure inside the fuel cell system. The operating conditions of the PEMFC system are listed in Table 1.  3 Copyright © 2014 by ASME In the hybrid system, the power output of the 12 kW inrack PEMFC was first converted to 192VDC and then connected to a 10 kVA, 208VAC L-L UPS system to supply AC power to the servers/load. Since the UPS system could only take 192VDC as input, the 48VDC output from the fuel cell was first converted to 192VDC via the converter. The UPS system converts the fuel cell power to conditioned power for the connected server/load. This configuration was selected for initial investigation of the battery and the fuel cell dynamic response characteristics, but it does not represent any preferred or optimal design of the type of hybrid fuel cell battery system envisioned herein.
The electrical properties of the full cells and the battery were measured using a Yokogawa WT1600 power meter with 50ms resolution. The fuel flow rate was measured using an Alicat hydrogen flow meter (M-250SOPM-D/CM) with 500ms resolution. Fuel cell system performances such as fuel cell coolant temperature, air flow rate, and anode pressure were monitored and recorded with 10s resolution via the fuel cell controller. The steady-state properties were measured at certain power set points, when the fuel flow rate and relevant temperatures were at equilibrium. The AC load bank used was Chroma programmable AC electronic load 63803, three load banks were connected in three phase parallel mode running under constant power mode with power factor=1.0. The type of servers used in this study was 750W HP Proliant SE326M1 server with two quadcore CPUs and 98GB of memory, total 9 servers were used in this study.

Steady-State Performance
The IV polarization curve of the PEMFC system tested in this study is presented in Figure 2   The steady-state efficiency curve of the PEMFC system is presented in Figure 3 with error bars showing the variation in the data. The efficiency is calculated using the following equation relative to the Lower Heating Value (LHV) of the hydrogen, and takes into account of all on-board parasitic loads.
System efficiencies were evaluated at steady state with various load (0, 1.5kW, 3kW, 4.5kW, 6kW, 7.5kW and 9kW) operating when fuel flow rate and coolant inlet/outlet temperatures were at equilibrium. As shown in Figure 3, when the fuel cell output about 50% of its rated power (10kW), the maximum system electrical efficiency will be achieved. Copyright © 2014 by ASME

Transient Performance
High resolution (sampling period=50ms) power profiles of the PEMFC system, the battery and the total hybrid system power output were obtained and presented in Figure 4 and Figure 5, to characterize the dynamic response performance of the in-rack PEMFC system. In Figure 4, the external load stepped from 4.5kW to 9kW at t=10 and in Figure 5 the external load stepped from 0kW to 9kW. In both cases, the battery in the hybrid system immediately supplies power to meet the step load and decreases after the fuel cell starts ramping up. During the smaller step load transient (from 4.5kW to 9kW), it took 3.2 seconds for the PEMFC system to ramp up and fully meet the 9kW external load, the peak power of the battery was 2.11kW and total energy needed from the battery was 0.00066kWhr. While during the larger step load transient (from 0kW to 9kW), it took 4.7 seconds for the PEMFC system to ramp up and fully meet the 9kW external load, the peak power of the battery was 7.52kW and total energy needed from the battery was 0.003kWhr. The testing results provide important insights on the battery sizing and cost optimization. The maximum discharge rate, cutoff voltage, maximum capacity and cost of the battery will be the dominate factors for sizing and optimizing the batteries in the hybrid system.
Fuel cell power overshoot led by the fuel cell voltage undershoot behavior is more prominent in the larger step load increase transient. The typical power overshoot behavior of PEMFC during step load transient is due to the overshoot effect of the current associated with an undershoot response of the voltage. Two main reasons for this phenomenon are 1) a temporary dry out on the anode side of the membrane during transient [13] and 2) air dilution/starvation at the reaction zone [14]. In the PEMFC anode, the relative humidity will decrease when the current suddenly increases during load change transient, which will result in dehydrating the membrane and increasing the membrane resistivity. Therefore the fuel cell is required to output additional power observed as the power overshoot behavior during load up transient. The resulted voltage undershoot is recoverable within a period of time constant characteristic of water back diffusion through the membrane. The duration of the power overshoot is therefore related to the redistribution of membrane water content [13]. In the PEMFC cathode, oxygen will be progressively depleted downstream along the gas channel during the load change transient, resulting in reactant gas dilution/starvation on the reaction zone. With large air dilution, the over-potential for the oxygen reduction within the catalyst layer will increase to sustain the operating current, leading to the voltage undershoot during load change transient [14]. The extra power from the overshoot will charge the battery slightly as shown in Figure 5.   A series of step load changes were applied to the hybrid system and the load profile is shown in Figure 6. After the fuel cell was turned on and running at steady-state with no external load (t=0), steps loads of 9kW (t=5), 7.5kW (t=10), 6kW (t=15), and 4.5kW (t=20) were applied to the system then repeated from t=35, with duration of 2 minutes for each of the step load. In Figure 7 to Figure 9, PEMFC system properties such as coolant temperature, cathode air flow rate and anode pressure are presented (sampling rate=10s).    As shown in Figure 7, the fuel cell coolant temperature rises sharply when the step load is turned on, while it cools down relatively slowly when there is no external load. Operating under higher external load, the steady-state fuel cell coolant temperature increases. During the test, the fuel cell was running most of the time with no load. It is noted that the coolant temperature gradually increased due to the series of step loads applied. The fuel cell system as tested increased the set point temperature of the internal cooling system as the load increased. The air flow rate showed in Figure 8 indicates that the air flow rate is adjusted stoichiometrically with the power so that the reactant and the product ratio can be fixed. Peaks can be observed for all the step load changes at the beginning of the transients. The speed of the cathode air blower is controlled internally by the fuel cell controller. As shown in Figure 9 the anode pressure varies within 1.5 psi ranges during the test.

Transient Performance with Servers
To evaluate the dynamic response performance of the PEMFC and the hybrid system, servers were also connected. In this configuration, servers were connected to the system output (120VAC, L-N) through a power distribution unit. Dynamic operation (turning on) of servers was performed and the response of the fuel cell power, battery power and total system output power are presented in Figure 10. As shown in the figure, the load of the servers changed almost instantaneously reacting to being turned on. It took the PEMFC system 3.2 seconds to ramp power up to 3kW while the battery supplied all the power needed in the first peak. To the power demand of the second spike, the PEMFC system ramped from 3.1kW to 5.0kW in 2.3 seconds, while much less power was drawn from battery than the first power spike. Eleven seconds after the servers were turned on, the PEMFC system could follow the server dynamics pretty well. It has been widely studied and reported that data center power consumption has both short and long term dynamic variations due to workload fluctuation and server on/off events. As demonstrated in this study, it usually takes several seconds for the fuel cell system to ramp power up as load changes. In situations when the server has to be cold rebooted, the fuel cell system must be able to follow sudden power demand increases, which may require an external battery. The transient responses of the PEMFC system and the battery presented in this study provide some important insights in the battery/energy storage sizing. Based upon our experiments with real servers, the only time that the server can cause large and rapid load changes is during the startup or shutdown processes. If the server power on and off events in a rack can be staggered over time, a single server sized battery can be shared by multiple servers in the rack. Unpredictable events such as rebooting a server in software or server software crashes do not cause any significant power change. This is because the electrical components are still powered in the reboot process. As a result, as long as single server spikes are met by the fuel cell/battery system, large load changing events are known to the management system. Data center workload also exhibits long term changes over days and weeks. However these changes are typically slower than the fuel cells ramping rates.

System Losses
The power outputs of the 12kW in-rack PEMFC system under various external loads is presented in Figure 11, with error bars showing the variation in the data. The sources and the percentage of the losses under various external loads are presented in Figure 12. The outputs profile shown in figure was evaluated at steady state with various external load (0, 1.5kW, 3kW, 4.5kW, 6kW, 7.5kW and 9kW) operating when fuel flow rate and coolant inlet/outlet temperatures were at steady-state. Fuel cell output power is the total power output from the fuel cell, calculated by multiplying the fuel cell voltage and current. The fuel cell power is converted to 192VDC via the DC/DC converter, and the DC system output power represents the power after the conversion. Noted that when there is no external load, the PEMFC system still output 0.97kW power to keep the fuel cell running and meet the parasitic loads. The major part of the parasitic loads is from the fuel cell exhaust blower. It runs at constant speed after the fuel cell is on, and consumes 0.71kW (400VDC) power constantly. The AC system output power is equal to the external load, and the differences between the DC output power and AC output power are the losses in the DC/AC inverter and the exhaust blower power consumption. The average DC/DC converter efficiency is 94.5%, and the average DC/AC inverter efficiency is 91.8%.
It is important to point out that with servers connected directly to the fuel cell 48VDC bus, the conversion and inversion losses in the system can be completely eliminated, and therefore higher system efficiency can be achieved.

SUMMARY AND CONCLUSIONS
In this paper, a distributed power architecture for fuel cell powered data centers to achieve high reliability and efficiency is proposed and demonstrated. The use of an in-rack 10kW PEMFC stack and system as the distributed power source to power a server rack is evaluated. By using this design, the power distribution system in the data center and the grid outside of the data center can potentially be eliminated. The steady-state performance and the transient response of the PEMFC system in response to real server loads have been evaluated and characterized. The PEMFC system is found to respond quickly and reproducibly to load changes directly from the server rack. Peak efficiency of 56.76% in a single server rack can be achieved. The contributions of system losses is also evaluated.
The ramp rate and dynamic response time of the system characterized in this study can be used to develop optimal control strategies to enable the dynamic load following capability. The battery transient response can be used to optimize the battery/energy storage size for the hybrid system. In addition, direct DC power from the fuel cell system could be applied to eliminate the capital cost and operating conversion losses from systems that use AC/DC conversion equipment. Reducing components in the energy supply chain will not only cut cost but reduce points of maintenance and failure, which will improve availability of the data center.