OK of 400 Gigabit Ethernet adds horsepower to data centers
Final approval of the 400 Gigabit Ethernet standard will provide data centers and cloud providers with the extra capacity they need to meet ever-rising data demands.
The IEEE 802.3ba standard for 40 Gigabit Ethernet and 100 GbE was completed in 2010, but it took only a short time before the IEEE recognized more speed would be needed -- and fast. The result, 400 Gigabit Ethernet, represents the latest advancement in the connectivity technology.
No one application generated the need for higher data rates. Rather, the number of internet users around the world has continued to grow, and new applications are constantly being developed. Adding to the traffic, web access has become more complex.
Originally, a request for information from a website gathered information from just one or a few sites. Now, every request generates multiple requests to create ads based on your interests and the other sites you've accessed recently. The result is increased traffic across the 100 GbE links currently supporting the cloud network core and the internet links between clouds. The advent of Gigabit to the home places additional traffic on service-provider networks.
Work on the standard for 400 Gigabit Ethernet, designated IEEE 802.3bs, began in March 2014. Later, in 2016, 200 GbE was added as an additional target. Requirements included maintaining the same Ethernet frame format used in previous specifications, as well as the same minimum and maximum frame size. Only full-duplex operation was required. The standard was finalized in December 2017.
The standard includes specifications for transmitting both 400 Gigabit Ethernet and 200 GbE over both multimode and single-mode fiber. The versions and their transmission standards vary:
400GBASE-SR16 uses 16 pairs of multimode fiber. Each pair carries a full-duplex 25 Gb stream for at least 100 meters.
400GBASE-DR4 transmits four 100 Gb streams over a distance of a minimum of 500 meters of single-mode fiber. It creates 100 Gb streams by using pulse-amplitude modulation (PAM4) to double the effective rate of a 50 Gb stream.
400GBASE-FR8 uses wavelength-division multiplexing (WDM) to multiplex eight frequencies over one pair of single-mode fiber. Each frequency carries a 50 Gb stream over 2 kilometers.
400GBASE-LR8 supports transmission over 10 kilometers. It uses PAM4 to create eight 50 Gb data streams that are each transmitted over single-mode fiber.
200GBASE-DR4 uses four pairs of single-mode fiber to transmit up to 500 meters. Each pair carries a 50 Gb stream that's created by using PAM4 to double the rate of a 25 Gb stream.
200GBASE-FR4 multiplexes four frequencies, each carrying 50 Gb up to 2 kilometers over one single-mode pair using WDM.
200GBASE-LR4 uses PAM4 to transmit four 50 Gb streams up to 10 kilometers over one single-mode pair.
Now, every request generates multiple requests to create ads based on your interests and the other sites you've accessed recently.
IEEE 802.3bs does not specify transmission over copper cable. Instead, IEEE 802.3cd, expected to be finalized later in 2018, will specify transmission of 200 GbE over 5 meters of twinaxial cable. It will be used primarily to connect servers to top-of-rack switches.
Multimode fiber is less expensive than single-mode, and it's widely used within data centers. But 400GBASE-SR16 requires a cable containing 16 fiber pairs for a total of 32 individual fibers. Reducing the number of fibers while retaining use of multimode is the goal of a new standard, known as wideband multimode fiber, or OM5.
A joint committee of the American National Standards Institute and Telecommunications Industry Association approved OM5 in June 2016. It specifies a technique using shortwave wavelength division multiplexing to carry 400 Gb over four pairs of multimode. In this approach, four 25 Gb streams are multiplexed over each pair of fibers.
Products trickling into the marketplace
Vendors will begin to roll out 802.3bs products in earnest later this year and in early 2019, although a handful of unofficially released components are now available. Cisco and Arista are among companies working with cloud vendors interested in upgrading their networks to 400 GbE, while Juniper and Ciena have been working on a field trial transmitting 400 GbE data over Verizon's Dallas backbone.
Also, open Ethernet vendor Mellanox is currently demonstrating its SN3200 series switch to customers, with support for 16 400 GbE interfaces. Mellanox will also use the new technology to enhance its InfiniBand products. And Broadcom just unveiled a new switch, the Tomahawk 3, which is capable of supporting 32 400 GbE ports.
Ultimately, even 400 Gigabit Ethernet will not be sufficient for web-scale data centers. Cloud-transport networks are continuing to grow rapidly. Once traffic levels threaten to overwhelm 400 GbE links, work will start on the next increment. A new standard will be developed, and the industry will again move on to higher and higher rates. And so it goes.
