kantver - Fotolia
Take data center efficiency to the max on three-phase power
Three-phase circuits provide considerably more power efficiency to data center equipment, but phase balancing can trip up deployments.
Three-phase power configurations, with expert phase balancing, are the key to maximum efficiency in the data center and avoiding unnecessary shutdowns.
Three-phase branch circuits to cabinets put hardware on 208-Volt service. This provides more power over fewer wires and offers better energy efficiency than single-phase, 120-V service. Any 20 kW or larger uninterruptible power supply (UPS) also uses three-phase power supply and distribution. Most data centers therefore have three-phase power somewhere in the facility, whether just at the power infrastructure or running to the cabinets of IT equipment.
Single-phase power
Incoming power from the utility is 240-V, delivered by two hot wires and one neutral.
For decades, it was standard practice to power cabinets at 120 V -- the same voltage and wiring scheme used for utility outlets in homes and offices.
Every receptacle is serviced by two wires -- hot and neutral -- plus a non-voltage-carrying ground wire (see figure 1). A full cycle of utility power goes through a 360 degree phase change, so the two hot legs are 180 degrees apart, which is known as single-phase power.
Big appliances, ones that do not get unplugged frequently, run on 240 Volts of single-phase power, using both hot wires to deliver twice the 120-V hot-to-neutral voltage, as shown in figure 1. The higher voltage delivers more power (wattage) with the same-sized wires and circuit breaker ratings.
Power outages
If you connect too large a load to a receptacle, the breaker will trip and interrupt the circuit. If several small loads connect to each receptacle, but the total exceeds the circuit breaker capacity, the breaker will trip, failing each receptacle on that circuit. It is possible for loads to be within limits on every branch outlet and lighting circuit, yet still overwhelm the power supply.
A shift in power
The North American utility standard is 240 V, but European and Asian data centers use 400 V
AC to DC conversions could be history with DC power
When power is at a premium, make your own
Each hot leg of 240-V utility power must have a main circuit breaker, but electrical code requires that the handles on both service legs be physically tied together. Therefore, if the total load on either hot leg exceeds the main breaker rating, it will trip both circuits, causing a total power failure. Both wires are connected by 240-V circuits, so draw the same amount of power from each leg. But 120-V service comes from only one of the hot wires and the neutral. Multiple 120-V receptacles are wired to each hot wire, so if you connect too many things to receptacles on the same leg of your incoming service, the main circuit breaker will trip both legs, not just the overloaded one. Running reasonably equal loads on both phases avoids this problem.
Is three-phase power to the cabinets the answer?
Commercial buildings mostly use three-phase power. More actual power is delivered through the building with the same-sized wires and circuit breakers, leading to significant electrical cost savings. The principles are the same as described above, but phase balancing is trickier.
Three-phase power service is delivered by three hot wires and one neutral. The voltage between any phase or hot wire and neutral is still 120 V, just like at home or older data centers. But three-phase power moves the legs 120 degrees apart. The voltage between any two phases or hot legs is 208 V.
Figure 2 shows several different possibilities derived from a three-phase system. The 208 V-power delivered by any two legs of a three-phase system is still called single-phase power. It is unnecessary to bring the neutral to data center cabinets with a three-phase system. With just the three-phase power wires, you have multiple 208 V-circuits. If the neutral is extended, then both 120 V- and 208 V-receptacles can be provided on the same power strip.
We get considerably more power from three-phase circuits than from single-phase designs. Without delving into electrical theory, the reason is that three-phase wattage is volts times amps times √3, rather than just volts times amps. Power factor is excluded; it makes little difference in modern computing hardware.
To translate design into real power load, see the final column of chart 1: 80% capacity. This is the maximum power that equipment is allowed by code to draw through a circuit breaker on a continuous basis. This is the real power that will go to data center equipment.
Computing hardware running at 208 V operates at higher energy efficiency than equipment running at 120 V.
Three-phase systems deliver nearly twice the power capacity of single-phase systems, but via fewer wires. Since most cabinets today are dual circuited, the difference in wiring demand doubles, escalating cost savings with three-phase power.
The wiring differences become extreme when powering ultra-dense cabinets (25 to 30 kW), especially with higher amperage circuits where conductor sizes are also bigger.
Phase balance is highly important with three-phase circuits. The phase with the highest load determines the maximum output of the data centers' UPS or power distribution units, so phases must be balanced to use the total power capacity. All three handles of a three-phase breaker are tied together, so if you overload one phase, it will trip the breaker and cut power.
Examples of power loading in data center cabinets
The goal for good data center power distribution is to have no more than a 5% load differential among phases. If you have a combination of 120- and 208-V devices in the data center cabinets, the 120-V circuits are easy to move around on a three-phase system. While 120-V circuits can make phase balancing easier, it's a poor reason alone to use them, because of the efficiency losses.
Calculating amperage per phase on unbalanced three-phase circuits is not straightforward. Metering is the best way to get actual per-phase amps in the data center.
Assume you have a cabinet full of the same type of servers, and one-third of them are plugged into every circuit group. The phases are automatically balanced as illustrated in Example 1. This cabinet will run on a 30-Amp, three-phase circuit since each phase is less than 24 Amps, or 80% of the 30 Amp breaker rating.
It's harder to phase balance if devices draw different power levels or if too many devices are connected to the same phases. Example 2 shows that when devices with different power requirements are randomly connected to a three-phase power strip, the loads go way more than 5% out of balance, and two of the phases are over the limit for a 30-Amp breaker. A larger circuit would solve that limit problem, but is actually unnecessary for the total load presented.
In Example 3, the data center operator moves two servers (highlighted in the chart) in an attempt to balance the phase loads. But when 208-V loads move, the load changes on only one phase. The loads are better matched on Phases A and C, but the load on Phase B is the same as before. Further, the differences among phase currents still significantly exceed 5% and Phase B is still above the 30-Amp circuit's maximum. Over time, the breaker could trip and shut down each circuit in the cabinet.
In Example 4, operators move two more servers (highlighted in the chart), with different power requirements, to achieve better phase balance. The result is only 1% differential and all three phases are now within the limits of a 30-Amp breaker, without changing the power equipment.
These examples show that you may need to move several devices around to achieve phase balance on a three-phase circuit.
About the author:
Robert McFarlane is a principal in charge of data center design at Shen Milsom and Wilke LLC, with more than 35 years of experience. An expert in data center power and cooling, he helped pioneer building cable design and is a corresponding member of ASHRAE TC9.9. McFarlane also teaches at Marist College's Institute for Data Center Professionals.
