High-Speed Interconnects for AI: Ethernet, InfiniBand, and the Path to 1.6T

The networking industry has moved well beyond the era when 10-gigabit links represented the bleeding edge. Today, AI training clusters routinely demand 400G and 800G connectivity per endpoint, and the first 1.6-terabit products are already entering deployment. Yet the terminology surrounding these advances is often muddled. Ethernet and InfiniBand are distinct protocols with separate roadmaps, and conflating them — or treating 10G as part of the “terabit” conversation — leads to confusion when specifying hardware. This article separates the two tracks and examines what network architects need to know as AI fabrics scale toward 1.6T.

Ethernet and InfiniBand: Two Parallel Interconnect Tracks

Ethernet and InfiniBand solve similar problems at the physical layer but diverge at the protocol level. Ethernet, standardized by the IEEE 802.3 working group, has evolved from its enterprise and cloud roots into a credible AI back-end fabric. InfiniBand, maintained by the InfiniBand Trade Association, offers a purpose-built architecture with native remote direct memory access, adaptive routing, and in-network computing features such as NVIDIA SHARP. A critical practical point is that the two protocols can share the same optical modules and connector form factors — a transceiver inserted into an InfiniBand switch activates InfiniBand mode, while the same module in an Ethernet switch operates as Ethernet. However, the switches themselves maintain separate protocols, and the speed generations are named differently: Ethernet uses round numbers like 400GbE and 800GbE, while InfiniBand uses labels such as NDR and XDR.

Ethernet’s Road from 100G to 1.6T

On the Ethernet side, the transition from 100G to 400G and 800G has been driven primarily by hyperscale cloud and now AI back-end networks. The Ethernet Alliance’s 2026 roadmap highlights the industry’s focus on 1.6T interfaces, Linear Pluggable Optics, and improved energy efficiency for AI-era deployments. The IEEE P802.3dj project, which covers 200 Gb/s, 400 Gb/s, 800 Gb/s, and 1.6 Tb/s Ethernet, has progressed through Standards Association ballot review and remains on track to define the next generation of high-speed Ethernet PHYs and MAC parameters.

For 400GbE deployments, the industry has largely settled on 100G-PAM4 electrical lanes, with OSFP and QSFP112 as the dominant form factors depending on whether the endpoint is a switch or an adapter. The 800GbE generation builds on eight 100G lanes or, in some implementations, four 200G lanes as SerDes technology advances. Looking ahead, 1.6T Ethernet will require 200G per lane PAM4 signaling and will rely on OSFP-XD or similar high-density form factors to manage thermal and electrical challenges.

InfiniBand Generations: EDR, HDR, NDR, and XDR

InfiniBand’s speed generations follow a different naming convention, and understanding the mapping between generation labels and actual port speeds is essential for procurement. The current mainstream generation is NDR at 400 Gb/s per port, deployed with NVIDIA Quantum-2 switches and ConnectX-7 adapters. NDR builds on the earlier HDR generation at 200 Gb/s per port, which remains widely installed in HPC clusters using 200G QSFP56 InfiniBand HDR transceivers. For legacy installations and cost-sensitive upgrades, 100G QSFP28 InfiniBand EDR modules continue to provide reliable 100 Gb/s per-port connectivity in existing fabrics.

The most important nuance in current InfiniBand deployments concerns NDR versus XDR at the 800G level. NDR is a 400G-per-port generation, and the 800G OSFP modules associated with NDR are twin-port designs: a single physical OSFP cage carries two independent 400G transceiver engines, delivering 2×400G optics and 800 Gb/s electrical to the switch. These twin-port 800G modules are used in NVIDIA Quantum-2 switches to connect switch-to-switch or to link with 400G endpoints. This is the configuration often marketed as “800G OSFP InfiniBand NDR” — it is an 800G module, but it operates as two 400G NDR links.

XDR, by contrast, is the 800G-per-port InfiniBand generation. XDR deployments use 1.6T twin-port OSFP modules on the switch side, where each cage provides two independent 800G XDR ports, and 800G single-port OSFP modules at the ConnectX-8 endpoint. The 400G OSFP/QSFP112 InfiniBand NDR portfolio reflects the form-factor flexibility of the 400G generation: OSFP for switch-side density and QSFP112 for space-constrained DPU and adapter endpoints.

Deployment Realities and Legacy Considerations

Deploying 400G and 800G interconnects involves more than matching speed labels. Power and thermal management become critical as transceiver dissipation rises. Twin-port 800G OSFP single-mode modules, for example, remain at approximately 17 watts across configurations, and high-density switches such as the Quantum-X800 can see system power reach several kilowatts when active optics are fully populated. Cooling variants — finned-top for air-cooled switches and flat-top for liquid-cooled systems or adapter cards — must be specified correctly.

At the lower end of the speed spectrum, 10GBASE-SR SFP+ and 10G SFP+ transceivers remain in service for server access and storage networks. While SFP and SFP+ share the same physical connector, the SFP+ specification was engineered for 10 Gb/s rates with tighter impedance tolerances. The 10GBASE-SR variant operates at 850 nm over multimode fiber for short-reach links up to 300 meters. These modules are not part of the terabit conversation, but they remain a practical starting point for organizations still building out 10GbE access layers before migrating to higher speeds.

The Road Ahead: 1.6T and Beyond

The next milestone is InfiniBand GDR at 1.6T, expected to reach volume production around 2027, while Ethernet’s 1.6T standard moves through IEEE ratification. For network architects planning AI clusters today, the key is to design for flexibility: choose switch platforms and cabling infrastructure that can accommodate the next speed generation without a full forklift replacement, and verify that transceiver form factors, cooling variants, and protocol activation match the specific switch and adapter in use. Ethernet and InfiniBand will continue to evolve in parallel, and treating them as interchangeable at the protocol level — or assuming that a module labeled “800G” means the same thing in both ecosystems — is a recipe for deployment surprises.

Leave a Reply

Your email address will not be published. Required fields are marked *