Data Centers’ Power and The Power of Data Centers
Data centers have become a frequent topic in articles and online discussions lately, often in the context of the rapid expansion of Artificial Intelligence.
However, do we truly grasp their fundamental importance and how deeply intertwined they are with our daily routines?
Let's illustrate this with some familiar examples. Every time you compose an email, send a WhatsApp message, or post on Facebook, you are, in fact, utilizing resources within a data center.
But what exactly is a data center? Why are they so important?
Essentially, it's a physical facility, typically a large warehouse, housing thousands of servers mounted in racks and connected to the internet backbone via high-speed links. These servers can host software for a single client or operate as multi-tenant environments, serving multiple users simultaneously.
Furthermore, whenever you access a company website – be it a news outlet, an online retailer, a medical reference site, or even your bank's portal – you are interacting with software running on one or more of these servers. The data you input embarks on a journey to a data center the moment you hit the "enter" key within your browser or mobile app.
Interestingly, the data center you connect with might not even be geographically close to the company's headquarters. Many European companies, for instance, host their websites in data centers located in the United States. Similarly, large corporations often maintain multiple data centers across different continents, and your query will likely be routed to the one nearest to your device to minimize response times, ensuring a smoother user experience.
Beyond the activities directly triggered by your actions, a significant amount of processing occurs in data center servers in the background. If you have an account with services like Netflix or Amazon, for example, there are continuous processes running on their computers to analyze user profile data. This analysis enables them to generate millions of personalized movie recommendation lists and tailored shopping suggestions. These are demanding processes that involve analyzing vast quantities of data and operate around the clock, 24/7, 365 days a year.
To efficiently handle the requests of millions, or even billions, of users within a reasonable timeframe, multiple servers are often combined into massive server pools, distributing the workload of incoming queries. Sometimes, a customer's request can be quite complex, requiring the system to execute multiple internal sub-queries involving various databases, customer profile validation, e-commerce transaction processing, and security authorization checks. It's not uncommon for a single customer query or transaction to interact with tens, or even hundreds, of different servers behind the scenes.
A typical data center is situated within a large warehouse and is organized into rooms. Within each room, servers are arranged in islands, and each island contains aisles or rows of racks. A single row can house between 8 and 16 racks. A rack is the physical structure where servers, along with network switches and other supporting devices, are installed.
The height of a rack is usually measured in Rack Units (RU), where 1 RU equals 1.75 inches (4.445 cm). Depending on the data center's room specifications, racks can be up to 52 RU tall. A standard data center server typically occupies 2 RU, although some "pizza box" servers are only 1 RU high. Generally, taller servers can accommodate a greater number of hard drives. A quick calculation reveals that a rack can typically host an average of 24 servers, although additional space is required for network switches and other necessary equipment.
These physical arrangements, along with the typical power consumption of a server, are crucial factors in determining the overall power capacity a rack is designed to provide, which is usually a maximum of 8 kilowatts (kW). This figure includes a safety margin, as data center management teams aim to keep the actual load within 60-70% of the rack's theoretical capacity or power budget. This cautious approach is well-justified, as you never want to risk needing more power than you actually have available at any given time, especially during emergencies. The goal, afterall, is to be able to maintain constant, uninterrupted service under any circumstances.
When you multiply this per-rack power figure by the number of racks in a row, the number of rows in an island, the number of islands in a room, and finally the number of rooms in a warehouse, you arrive at the total power a provider must be capable of supplying to operate the entire facility.
In reality, however, the calculation often works in reverse. The total power a provider can deliver to a data center is not limitless but is constrained by the regional power infrastructure. Therefore, data center designers must first determine the available power budget at each specific location and then work backward to define the specific power allocations down to the individual rack level. I've personally observed data centers with rows operating at half capacity due to limitations in the power supply, hindering their ability to scale up and resulting in inefficient use of valuable real estate.
We’re still discussing power consumption in terms of Megawatts (MW) here. In fact, the capacity of local energy providers is a critical factor in deciding where to deploy a data center. The LinkedIn data center in Hillsboro, Oregon, for example, is sized for a power consumption of up to 8 MW.
Beyond power availability, other key logistical considerations include seismic and flooding risks, water availability, and robust internet backbone connectivity. From an economic standpoint, the cost of electricity and water, as well as potential tax advantages, are also significant factors.
However, the landscape of data center requirements is being dramatically altered with the rise of AI and the associated demand for powerful Graphics Processing Units (GPUs).
The following chart illustrates the recent and projected power demands per single server rack in a data center:
Estimate for 2018-2022: 5kW (typical of a single apartment)
Average in 2023: 6kW (enough to power a 3-4 bedroom household)
Average in 2024: 8kW (comparable to a large household)
One rack of NVIDIA H200 GPUs (2024): ∼50kW (equivalent to a small business or 35 apartments)
NVIDIA GB200 NVL72 (2025): ∼130kW (enough power for a large HVAC unit or an electric motor used for machinery)
NVIDIA Rubin Ultra NVL576 (2027): 600kW (typical of large industrial facilities, manufacturing plants, and even some commercial buildings)
The situation appears to be escalating rapidly.
We need to plan for a staggering 75-fold increase in power requirements between now and 2027. This implies that a typical standard data center with a 10 MW capacity will need to be upgraded to a 750 MW facility. Furthermore, there are already plans on the table aiming for data centers with a 1 Gigawatt (GW) capacity. This increase is both rapid, as well as exponential, which has a huge impact on energy needs.
Considering the steady increase in electricity demand from residential, commercial, and industrial sectors (think of the growing number of EV charging stations and the increasing deployment of robotic devices in factories), coupled with the climate uncertainties of recent years, adding this immense additional strain on the power infrastructure is a critical concern. We all remember the major power crisis in Texas in the winter of 2021, where exceptionally cold weather and the increased demand for heating led to the shutdown of the power grid for weeks, with more than 4.5 million homes and businesses were left without power, causing at least 246 deaths.
This is a significant reason why data center designers are exploring the construction of "after the meter" power generation stations. These are private power plants (utilizing natural gas, solar energy, or a combination of both) built specifically to supply the energy needs of a unique data center, therefore providing an independent power source that would not impact the currently established grids.
This shift represents an incredible surge in data center infrastructure costs, bringing with it a host of new challenges to address and a demand for new expertise and skills.
What about cooling, though?
Cooling is inextricably linked to power consumption. The operation of electronic components on computer boards generates heat, and these components can only function within a specific temperature range. Exceeding this threshold leads to automatic system shutdowns and, therefore, a loss of service which is the ultimate failure scenario for any data center.
Traditional data centers relied on air-based cooling systems to dissipate the heat generated by the servers in the racks. Enormous air conditioning (HVAC) units were installed to maintain the rooms at the lowest possible temperatures. Cold air was forced underneath raised technical floors within the data center rooms and then channeled through special perforated tiles, entering the aisles and cooling the racks from the bottom up. However, these HVAC systems themselves are significant consumers of electricity.
Several years ago, liquid-based cooling solutions for servers began to emerge. One early approach involved specialized rack doors that functioned like refrigerators. Only the doors were connected to the cooling liquid pipes, while the servers themselves remained standard, requiring no special modifications. Monitoring software remotely controlled the door's efficiency and the temperature of the cooling liquid. Although the cost per door was around $10,000, this solution offered the advantage of reducing the need for large HVAC generators (both in terms of the cost of the units and the floor space required to house them) and eliminated the need for raised technical floors in the data center rooms. However, it necessitated the installation of an extensive piping system throughout the facility to circulate the cooling liquid.
As the demand for more powerful servers and faster network connectivity increased, the core components within servers began to operate at higher speeds (clock frequencies) and incorporate a greater number of processing cores within a single chip. Similarly, network controllers evolved from 1 Gigabit per second (Gb/s) to 10 Gb/s, 40 Gb/s, 100 Gb/s, and even 400 Gb/s. This increase in speed translates directly to higher power consumption and, consequently, more heat that needs to be dissipated. A similar trend occurred with the increasing capacity and density of hard drives.
This heightened demand for cooling led to the development of more advanced and radical liquid cooling solutions. Two main approaches emerged: full immersion liquid cooling and on-chip liquid cooling.
The full immersion cooling solution involves submerging the entire server boards, including storage devices, in large tanks filled with a dielectric (non-conductive) liquid. While this solution has a significant impact on the overall data center design, it doesn't drastically alter the design of the server boards themselves. Instead of vertical racks, servers are now deployed in horizontal tanks. This approach does, however, significantly affect the maintenance procedures for data center operators, who must remove the boards from the tanks before performing any maintenance work.
The on-chip liquid cooling solution brings the cooling liquid directly to the hottest components on the server board. This is achieved by integrating small channels within the heat sinks that are typically installed on top of these critical, heat-generating devices, thereby directly removing the heat. This method requires pipes (both for cold liquid inflow and hot liquid outflow) to reach the servers, pass through the chassis, and then run across the components on the board.
With this method of cooling, in addition to the standard cabling for data and power connectivity, servers need to provide connectors for attaching these cold and hot liquid pipes. Typically, a cooling unit is also installed on top of each rack to cool the liquid returning from the servers and circulate it back down. This necessitated a complete redesign of the server chassis to accommodate these new requirements, as well as a new design for the racks to allow the pipes to run up and down, connecting the refrigeration unit and the servers. A potential drawback of this approach is that pipes and their connectors are mechanical devices and are therefore susceptible to leaks over time, which is an added risk factor.
Will either of these existing liquid cooling solutions alone be sufficient for the new generation of high-power, high-heat devices?
It's unlikely.
Each approach has its limitations: air flow speed and the boiling point of the liquid. Even today, high-density GPU-rich servers often utilize a hybrid approach, combining air cooling (around 30%) with on-chip liquid cooling (around 70%).
Liquid cooling introduces a significant new requirement: the need for substantial amounts of public water for this purpose. Beyond the general availability of water, we are increasingly living in times of more frequent and severe droughts, leading to intense competition for the allocation of public water resources between agricultural and industrial uses.
In summary, the data centers of the near future will have demands that the current infrastructure can scarcely meet. Even if upgrades were feasible, they would require a significant timeframe (5-8 years), a timeline that the rapid advancement of AI doesn't afford.
Consequently, we are beginning to see solutions based on "after the meter" power plants, such as the one planned for the massive new data center under construction in Abilene, Texas, as part of the Stargate project, as well as alternative and innovative offerings for off-the-grid data center deployments.
Companies are already offering such solutions, utilizing liquid hydrogen as fuel and fuel cells as power generators. EdgeCloud Link (ECL) is one such example. This is no longer just a theoretical concept or a lab experiment. ECL has an operational pilot data center at their headquarters in Mountain View, California, and a first multi-MW commercial data center currently under construction in Texas. They offer a modular design, with units ranging from 1 to 2 MW each, to accommodate various scaling needs. They can adapt existing traditional facilities or assist in entirely new designs and implementations.
The significant advantage of such a solution is its minimal environmental impact. It would not necessitate the construction of additional new power plants (which typically burn fossil fuels like oil, gas, or even coal) and would not require any public water for cooling the servers, as water is a byproduct of the fuel cell's operation and is therefore self-produced as part of the process.
Thus, instead of further burdening the already strained power infrastructure and consuming vast quantities of public water for server cooling, this solution offers a self-contained design that can scale to meet the burgeoning power demands driven by the AI revolution.
Is this the only viable solution?
Perhaps it is, at least for the immediate future.
Alternative geo-thermal or hydroelectric sources are geographically constrained. Small nuclear reactors still face the persistent challenges of managing the disposal of spent radioactive fuel and addressing public concerns about their safety, and in any case, their deployment requires considerable time.
Therefore, in my opinion, hydrogen and fuel cells represent the most promising option we currently have to address the immediate need for increased power in our data centers. Best of all, this technology is already available, all it would take is the right investment, both on a project level as well as a monetary level, to make it the new standard across all data centers.