Liquid-cooled servers in data centers

With the expansion of arithmetic power scale, data center power consumption also grows rapidly. According to the China Data Center Industry Development White Paper (2023), it is expected that by 2025, the electricity consumption of national data centers will reach 1.2×1011kW-h, and the total carbon dioxide emissions from national data centers are expected to reach 10,000 million tons, accounting for about 1.23% of the total national emissions. In order to implement “carbon peak, carbon neutral”, the state issued a series of policies to promote energy saving and consumption reduction in data centers, PUE index requirements gradually to 1.3, 1.25 or less rapid evolution, the current data center traditional cooling technology gradually unable to meet the requirements.

At the same time, with the continuous increase of single-chip arithmetic density, air cooling has gradually become a bottleneck, and the emergence of AI servers and other high power consumption equipment has also exacerbated the phenomenon of remaining cabinet space. The introduction of liquid cooling technology has become more and more urgent under the two-way pressure of the national dual-carbon policy requirements and the continuous increase of chip arithmetic density.

1
Advantages of Liquid Cooling Technology


Compared with the traditional air-cooling technology, liquid cooling technology can significantly improve the cooling efficiency and business reliability, and the environment is well adapted to reduce land occupation.

(1) Reduce the energy consumption of equipment and help save energy and reduce carbon. In the era of digital economy, the rapid growth of arithmetic power demand, coupled with the widespread application of 5G, has led to the booming development of information infrastructure, and data center power consumption and carbon emissions have also grown significantly. With liquid cooling technology, data center PUE can be reduced to about 1.1 to 1.2, which can effectively reduce energy consumption and carbon emissions. Taking 100 20kW liquid-cooled cabinets as an example, the PUE is reduced from 1.45 to 1.15, and the annual electricity consumption can be saved more than 1×107 kW-h, the electricity cost is saved more than 7 million yuan, and the carbon emission is reduced by 6,000 tons. At the same time, the liquid-cooled environment can reduce or remove the fan, further reducing server energy consumption.

(2) Extend equipment life and reduce operation and maintenance pressure. According to the chip 10℃ law, every 10℃ increase in temperature reduces component life by about 50%. The use of liquid cooling can effectively reduce the CPU junction temperature by more than 10°C (junction temperature is the actual operating temperature of semiconductors in electronic equipment), extending server life. Compared to traditional air cooling, immersion liquid cooling removes the fan, reduces noise and dust, and has a low thermal failure rate. Data shows that the hard disk failure rate in a submerged liquid-cooled environment is about 50% lower than in an air-cooled environment. The lower failure rate and longer life of the server can effectively improve the stability of business operation and reduce the pressure of operation and maintenance.

(3) Good environmental adaptability and high rack mounting rate. Traditional air-cooled cabinets have high environmental requirements and are greatly affected by the external environment and temperature, while liquid-cooled cabinets have low environmental requirements and can be deployed in areas with large temperature differences to obtain a more stable PUE. liquid-cooled cabinets have a high installed density, which is about 3 to 4 times that of traditional air-cooled cabinets, and saves about 75% of the area of the server room under the same arithmetic power. Liquid-cooled server room can avoid deployment of air conditioning, reduce the number of chiller units and save server room space.

2
Liquid-cooled technology route



Air-cooled servers use air as the heat transfer medium, while liquid-cooled servers use industrial water or liquid fluoride as the heat transfer medium. Due to the differences in density, specific heat capacity and thermal conductivity, the heat dissipation capacity of liquid with the same flow rate can be increased up to 3500 times compared with that of air. The efficient cooling method of liquid cooling can significantly increase the installed density of servers and reduce the occupation of rack and server room space.

According to the way of contact between the coolant and the heat source, liquid cooling technology can be divided into two categories: non-contact liquid cooling and contact liquid cooling.

2.1

Non-contact liquid cooling



Non-contact liquid cooling mainly refers to cold plate liquid cooling, through the liquid flow in the cold plate to take away the heat of the heating element (CPU/GPU/VRD/DIMM, etc.), to realize the heat dissipation of some components of the server. The liquid in cold plate liquid cooling is not in direct contact with the heat source.

The specific heat capacity and thermal conductivity of the liquid is good, the cooling efficiency is high, cold plate liquid cooling compared to the traditional air-cooled cooling technology can achieve 60% to 90% reduction in energy consumption, data center PUE value can be reduced to about 1.2. At the same time, the cold plate liquid cooling server deployed in traditional cabinet mode, the current server room supporting and server transformation difficulties and costs are small, but because it is only part of the server in the high heat generation components using liquid cooling heat dissipation, servers still need to configure a small number of fans, server room also need to configure air conditioning and other non-liquid cooling components in the server for air cooling heat dissipation.

Heat pipe liquid cooling also belongs to the non-contact liquid cooling, the use of heat pipes to transfer the heat of the CPU and other heating components to the server outside, and then taken away through the circulating water, mainly using water, Freon and alcohol liquids as the refrigerant medium, is currently less used.

2.2

Contact liquid cooling

The liquid of contact liquid cooling is in direct contact with the heat source, which can be categorized into submerged liquid cooling and spray liquid cooling according to the deployment mode of the server and the form of the cabinet.

Submerged liquid cooling is to completely submerge the server in the coolant, the server and the coolant are in direct contact, and the heat is taken away through the circulating flow or evaporation and condensation of the coolant. Coolant is mostly mineral oil, silicone oil, fluoride fluid and other non-conductive, non-corrosive liquid.