10GbE, or 10-Gigabit Ethernet, has been gaining traction for a number of years now. Although the standard (IEEE 802.3ae) is nearly 20 years old, the first 10 years were dominated by the industry building out its 1GbE networks, due primarily to cost and cabling issues. Over the last 5-10 years however, its use is transforming enterprises, data centers, NAS (network-attached storage), and virtualized environments, and this trend will certainly continue, as its adoption drives more and more use cases and environments where 10GbE is rapidly becoming indispensable. To facilitate this technology, 10GbE routers are being installed in more and more locations. Let’s take a look at 10GbE routers, where and why they’re needed, and some considerations if you’re considering jumping into the 10GbE world. But first, a little trivia…
What U.S city was the first to offer 10GbE Internet connections? It must be in Silicon Valley, right? Wrong! Salisbury, North Carolina, population 33,000, became the first U.S. city to provide 10Gb internet back in 2015. A local private college was the first customer to receive the service. The fact that a municipality, along with a local college, were the first (wide-scale) implementation of 10GbE is interesting and informative. In general, 10GbE is not appropriate for individual consumers (at least, not yet although hardcore home networking aficionados are using 10G switches – but routers, not so much). But metropolitan areas and colleges are a great fit as 10GbE can be deployed across the MAN (metropolitan area network) and the CAN (campus area network) efficiently.
10GbE, built around Layer2 Ethernet technology, is still primarily concerned with treating pain points (aka bandwidth congestion and high latencies) associated with LANs, local area networks, and perhaps access links to WANs, but not WAN cores or backhauls in general. 10GbE operates in full-duplex mode only and supports data transfer rates of 10 gigabits per second for distances up to 300 meters on multimode fiber optic cables and up to 40 kilometers on single mode fiber optic cables.
On the 10Gb routing side of the equation, the use cases of 10Gb routers are:
On the 10Gb switching side of the equation, there are many natural use cases for 10GbE:
Each of these scenarios/applications greatly benefits from 10GbE connectivity to handle the local traffic. Largely the routers will be located in data centers (physical or public/private/hybrid clouds) and SAN and NAS computing environments, where massive amounts of data from multiple users and applications converge. Let’s look a little more closely at a few of the above use cases.
Data centers have been a natural breeding ground for 10GbE. Two critical drivers of the requirement for a 10x improvement over legacy 1Gb networks have been large-scale adoption of virtualization, and cloud-based infrastructures. Servers have been getting faster and faster over the last decade (server CPU throughput has historically doubled every 18 months) which allows more virtual machines running on each server. Each VM competes for I/O on each server. More and more data must be processed, analyzed, shared, stored, and backed-up. Adoption of cloud-based services further increases the demands on VMs, servers, and data centers. 1GbE connectivity has been struggling to keep pace, and adopting 10GbE results in several immediate advantages:
Wireless Access Points
Wireless access points (WAPs) are another area where 10GbE has been critical. Wireless access points have exploded in use over recent years and the demand on their resources has overwhelmed 1GbE connectivity. There have been dramatic increases in the number of users per WAP and also the amount of data being shuffled back and forth per user. And with the relevant wireless spec, IEEE 802.11ac, stating a theoretical maximum throughput of 6.933 Gbps, 10GbE is required to achieve these speeds. While this may involve a significant effort in re-cabling, it is clearly necessary in the long run.
Another interesting space where 10GbE is gaining traction is the personal, desktop computer space. While 1Gb connectivity has been largely satisfactory for most of this market, there are certain, heavy users who could benefit from the 10x improvement afforded by 10GbE. And with the overall cost-of-ownership continuing to drop, it has become feasible for individual (power) users to upgrade their systems to 10GbE. Some of the fields which could really use the upgraded performance include:
NAS and SAN Implementation
10GbE also plays a major role in NAS (network-attached storage) and SAN (storage area networks) implementations. Ethernet storage protocols dominate much of the new virtualized environments, including NAS protocols such as Network File System (NFS), Server Message Block (SMB), and Common Internet File System (CIFS), and also SAN protocols including Internet Small Computer System Interface (iSCSI), and Fibre Channel over Ethernet (FCoE). As physical servers continue to implement multiple VMs per box, the I/O bandwidth required increases accordingly.
10GbE Routers Interfaces
10 Gigabit Ethernet maintains backward compatibility with existing Ethernet standards, e.g. 10/100/1000BASE-T (1 Gigabit ethernet over twisted copper pairs). So these routers can be easily used to upgrade existing Ethernet networks incrementally, addressing choke points in order of importance. Incorporating 10GbE into your existing network requires 10Gb-capable NICs (network interface cards) on targeted servers or workstations, as well as 10Gb-capable switches.
As far as interfacing to the 10GbE router goes there are two choices:
Your particular deployment scenario will determine which interface option is best for you. And hybrid solutions may be a great approach, as you can upgrade your existing network, piece by piece.
Some general guidelines:
Salisbury, North Carolina Revisited
Back to why Salisbury, North Carolina is a good fit for a 10GbE upgrade. With a physical size of about 18 square miles, the city of Salisbury can easily deploy fiber optic cabling from a central location and reach the entire town. And by using a local college as a first client, the college could relatively easily and cheaply upgrade their multi-campus data centers and LANs without any new cabling required. The campuses themselves would then be linked using the underlying fiber-optic backbone from the service provider.
40GbE, 100GbE and Beyond
10GbE is currently transforming many aspects of our digital universe, including data centers, wireless access points, ethernet-based storage solutions, and many others. The 10-to-1 consolidation of resources combined with the 10x improvement in bandwidth and latency allow enterprises to keep pace with the surging demand for speed and low latency across many sectors. And while this is a huge improvement over 1Gb networks, there continues to be a demand for higher and higher speeds.
In 2010, the IEEE ratified IEEE 802.3ba-2010 which establishes standard for 40GbE and 100Gbe. Over the last few years the technology has started gaining traction and will continue to do so with increasing speed as the associated costs continue to drop. The IEEE task force emphasized that these increased bandwidths are needed for:
These ultra-high speeds are currently being used in service provider networks, and the technology is sometimes called “Carrier Ethernet” which emphasizes that widespread use among enterprises is still in its infancy, although its adoption in data centers is accelerating.
Rob Stone, Mushroom Networks, Inc.
Mushroom Networks is the provider of Broadband Bonding appliances that put your networks on auto-pilot. Application flows are intelligently routed around network problems such as latency, jitter and packet loss. Network problems are solved even before you can notice.
© 2004 – 2021 Mushroom Networks Inc. All rights reserved.
Let’s chat. Call us at +1 (858) 452-1031 or fill the form:
Download your copy of rare tips and tricks for a better WAN. Get your free copy today!
We respect your privacy.