For most of data center history, the network was the least interesting part of the infrastructure stack. It moved packets. It connected servers to storage and storage to the internet. Engineers designed it for reliability and redundancy, not for raw performance. A well-configured leaf-spine fabric running at 10G or 25G per server port was more than sufficient for the workloads it served. Enterprise applications, database traffic, web serving, and virtualized compute all generated traffic patterns that conventional network design handled comfortably. The network was infrastructure in the most literal sense — present everywhere, noticed by nobody, and rarely the constraint that determined what the facility could actually do.
That era ended when GPU clusters arrived at commercial scale. A single Nvidia DGX H100 server contains eight GPUs, each connected to a 400G network interface, generating up to 3.2 terabits of aggregate bandwidth per server. A thousand-GPU training cluster built from 125 such servers requires a fabric capable of sustaining over 400 terabits of all-to-all traffic with near-zero packet loss and microsecond latency variance. The network is no longer background infrastructure. It is a primary determinant of how efficiently an AI training cluster performs, and the engineering decisions that govern its design have direct consequences for the economics of AI compute delivery.
How GPU Communication Breaks Conventional Network Assumptions
The traffic patterns that AI training generates differ from conventional data center traffic in ways that conventional network design is fundamentally ill-equipped to handle. Traditional enterprise traffic flows predominantly north-south, moving between clients and servers or between servers and storage in patterns that are largely independent of one another. AI training generates all-to-all east-west traffic, in which every GPU in a cluster must communicate simultaneously with every other GPU during the gradient synchronization operations that distributed training requires. This all-reduce communication pattern creates simultaneous demand for bandwidth across every link in the fabric, exposing any point of congestion instantly and catastrophically.
Packet loss in a conventional data center network is a manageable inconvenience. TCP retransmission handles dropped packets transparently, and the performance impact of occasional loss is negligible in workloads that tolerate latency variation. Remote Direct Memory Access, the protocol that GPU interconnects use, has no built-in retransmission mechanism. A single dropped packet in an RDMA flow stalls the entire training operation across the affected GPUs until the condition resolves. The difference between a network optimized for AI training and one that is not is not a marginal performance gap. It is the difference between a cluster that achieves its theoretical compute utilization and one that delivers thirty to sixty percent less training throughput on identical hardware.
The InfiniBand to Ethernet Transition
InfiniBand dominated AI cluster interconnects for most of the period when large-scale GPU training first became commercially significant. Its credit-based flow control mechanism provided inherently lossless transport without the configuration complexity that making Ethernet lossless requires, and its latency characteristics suited the tight synchronization demands of distributed training. For operators deploying their first large GPU clusters between 2020 and 2023, InfiniBand was the default choice that eliminated the risk of network-induced performance degradation.
The economics of InfiniBand at scale are significant. A 512-GPU cluster built on InfiniBand NDR requires approximately $2.5 million in networking hardware, compared to roughly $1.3 million for an equivalent Ethernet configuration. That cost differential compounds across the gigawatt-scale AI campuses that hyperscalers and major AI operators are now developing. Meta’s decision to train its Llama 3 models on RoCEv2 over Ethernet at a scale of 24,000 GPUs validated that properly configured Ethernet delivers equivalent training throughput to InfiniBand for the workloads that represent the majority of production training activity. By 2025, Ethernet had captured the majority of AI backend network deployments, and the Ultra Ethernet Consortium is building next-generation specifications designed to match InfiniBand’s native RDMA capabilities within an open ecosystem framework.
What Lossless Ethernet Actually Requires
The transition from InfiniBand to Ethernet for AI interconnects does not represent a simplification of network design. It represents a transfer of complexity from proprietary protocol stack management to precise configuration of Ethernet mechanisms that were not originally designed for the zero-loss requirements of RDMA traffic. Priority Flow Control, Explicit Congestion Notification, and careful traffic class separation must all function correctly and in coordination for Ethernet to deliver the lossless transport that AI training demands. Each of these mechanisms requires configuration choices that differ fundamentally from the defaults appropriate for conventional data center traffic.
The consequences of misconfiguration are severe and non-obvious. Enabling Priority Flow Control globally rather than only for RDMA traffic classes creates cascading pause storms that can reduce Ethernet performance to a fraction of its theoretical capacity. Applying Explicit Congestion Notification thresholds appropriate for conventional HTTP traffic to AI all-reduce operations creates marking behavior that degrades gradient synchronization performance under exactly the high-load conditions where correct behavior matters most. These are not edge cases or theoretical risks. They are failure modes that operators deploying Ethernet for AI interconnects encounter regularly, and their resolution requires network engineering expertise that the conventional data center workforce does not uniformly possess.
Scale-Up Versus Scale-Out Architecture
The distinction between scale-up and scale-out networking architectures has become central to how AI infrastructure planners think about cluster design at the largest deployment scales. Scale-up architectures, exemplified by Nvidia’s NVLink, create extremely high-bandwidth, low-latency connections between GPUs within a single server or rack, enabling tight coupling that suits the synchronization requirements of the largest frontier model training runs. Scale-out architectures, implemented through InfiniBand or Ethernet fabrics, extend connectivity across racks and across facilities, enabling clusters to grow beyond what single-rack integration can support.
The relationship between these two architectural layers is not competitive but complementary, and understanding how they interact determines the design choices that govern cluster performance at scale. A training run that fits within the NVLink domain of a single NVL72 or NVL144 rack experiences dramatically lower communication overhead than one that must synchronize across a scale-out fabric. As frontier model sizes grow and training runs increasingly exceed what single-rack NVLink domains can contain, the performance characteristics of the scale-out fabric become the binding constraint on training efficiency. The operators who design their scale-out fabrics around the specific communication patterns of their largest planned training runs achieve utilization rates that generic fabric designs cannot match.
Network as a Strategic Infrastructure Decision
The power efficiency of faster data center networks is also becoming a more prominent factor in network architecture decisions as AI facilities scale. Network infrastructure consumes power at rates that compound across the port counts required for large GPU clusters. As a result, the energy efficiency of switch silicon has become a meaningful variable in the total power budget of AI facilities. The operators who treat network architecture as a strategic infrastructure decision rather than a procurement exercise are making choices that affect facility economics across the operational life of their AI infrastructure investments.
As AI inference workloads grow, the network topology requirements of inference differ significantly from those of training. Consequently, facilities whose network architectures accommodate both workload types without fundamental redesign will hold structural advantages over those optimized for a single workload class. Furthermore, the network topology decisions being made in AI data centers today are not just technical choices. They are infrastructure commitments with decade-long consequences.
