Optical networks: Core network design best practices

Optical networking is the only relevant Layer 1 technology today in the network's core except in very unusual markets or geographic conditions where terrestrial microwave may still be deployed. While core optical network deployments in some areas may be literally indistinguishable from metro fiber, there are other networks where the key requirements are totally different. Thus, the first question in optical network design and deployment for core networks is the nature of the network itself, and how core and metro requirements might differ.

The "core" of a network is a place where aggregated traffic moves among on- and off-ramps. Because the traffic is highly aggregated and thus represents thousands or millions of user relationships, core network nodes are likely to have traffic destined for virtually all other core nodes, meaning that the nodes are highly interconnected. This may contrast sharply with metro networks, where "preferred topologies" often involve simply connecting serving or edge offices with points of presence (POPs) for connection to the core -- a star topology instead of a mesh.

The aggregation of traffic onto the core creates another difference in core optical networks: It is unlikely that a single network user will contribute a large portion of total traffic, and thus adding new users will generally not change the core significantly. In contrast, a large metro user may require reconfiguration of network bandwidth to accommodate the traffic. For this reason, and because of the "mesh" factor above, reconfigurability is likely to be less of an issue in core networks.

Core networks are also typically immune from the need for fast failover, the 50 ms optical alternate routing available with SONET. Ring configurations using fully redundant fiber paths are harder to create and more expensive to maintain in core networks, and so resilience is typically left to the electrical layer.

The final difference is that of geographic scope. A large metro network might span 50 to 100 km; a large core optical network can circle the globe. This long reach necessarily means that core fiber may have to span great distances without intermediate repeaters, including submarine environments, deserts, etc. Thus, ultra-long-haul fiber technology is often critical in core networks. The greater geographic scope of the core network also means getting craft personnel to an area to fix a problem may require days or weeks, and so it is critical to have some form of backup plan and to reduce outages as much as possible through design.

One issue that core and metro networks share is the issue of synchronous or circuit-switched traffic. Where PSTN calls and T1/E1 lines are to be supported over the core, it will likely be essential to utilize SONET/SDH transport for at least some of the optical paths to provide for synchronous end-to-end delivery. SONET/SDH services over global distances also require very accurate clocking to insure that bit errors are not created through "clock slips." These SONET/SDH trunks can either use the standard 1310 nm wavelength or one of the 1550 nm WDM wavelengths. Packet traffic does not require SONET/SDH, but many core network operators continue to use some SONET/SDH ADMs and switching in the core to preserve the option of circuit switched services.

A "pure packet core" can be made up of single-wavelength or WDM fiber, and thus it may be possible to create a virtual optical topology that approaches a mesh to avoid electrical handling. However, routers often have "adjacency problems" when installed in a full physical mesh, creating very long convergence times in the case of a failure. This, combined with the fact that reconfigurability in the core is often not a major requirement, means that core networks are more likely to use very high-speed fiber paths (OC-768, or 40 Gbps, for example) if the economies of these single electrical interfaces are better than the sum of the cost of WDM and a larger number of slower interfaces (4x10Gbps Ethernet).

The router adjacency issue is an example of an important point in core fiber design, which is that the needs of the electrical layer and even the service goals must be considered. Current trends in service provider Ethernet, spearheaded by work in the IEEE and the Metro Ethernet Forum are making Ethernet a strong candidate for core network deployment both to provide flexible virtual routes for higher-layer protocols like IP and to serve as the basis for actual customer services. This approach allows operators to create meshed optical networks for resiliency and add packet routing and even multicasting without creating additional router adjacencies.

A major consideration in optical core networking is the location of major points of service interconnection. The larger a provider core network is in terms of geographic scope, the more likely it will interconnect with networks of other providers, especially for local access in other geographies. These interconnection points are obviously both major traffic points requiring special capacity planning and points of major vulnerability. No interconnection point with another operator should be single-homed in fiber connection, nor of course should metro connections with the core provider's own metro infrastructure be single-homed.

The final point in core optical design is the management framework. Core networks carry aggregated traffic from millions of users, and failures will result in a flood of customer complaints. In addition, optical failures will trigger an avalanche of faults at the higher protocol layers, generating so many alerts that the network operations personnel may be overwhelmed. Many operators have insufficient integration between packet and optical layer management, and this increases vulnerability to alert storms and also makes customer support personnel less likely to have ready answers to complaints. The best optical core is no better than the operator's ability to manage it properly.

About the author: Tom Nolle is president of CIMI Corporation, a strategic consulting firm specializing in telecommunications and data communications since 1982. He is a member of the IEEE, ACM and the IPsphere Forum, and the publisher of Netwatcher, a journal in advanced telecommunications strategy issues. Tom is actively involved in LAN, MAN and WAN issues for both enterprises and service providers and also provides technical consultation to equipment vendors on standards, markets and emerging technologies.