The director switch: Basic SAN switch, plus scalability and high availability

Directors defined

The fundamental architectural difference between FC switches and FC directors is that FC directors are designed and built to scale up, and to provide high bandwidth and high availability. Today's FC directors are built with a blade-type design, so that additional ports can be added as needed by slotting an additional blade. The current crop of directors can scale up to several hundred Fibre Channel ports in a single unit by adding blades that contain various increments of ports.

The Brocade DCX Backbone family of directors can be deployed in single- or dual-chassis configurations, with support for up to 384 FC ports in the single chassis or a maximum of 768 FC ports in the dual chassis. A smaller version in the Brocade DCX line, the DCX-4S, scales up to 192 ports. Ports for the Brocade DCX Backbone directors are added by inserting 16-, 32- or 48-port blades that support 8 Gbps Fibre Channel. The Brocade 48000 Director is the previous-generation director that's available in a single-chassis-only configuration; however, it doesn't have the local switching capabilities of the DCX Backbone.

Cisco MDS 9500 Series Multilayer Directors are available in three models, the 9506, 9509 and 9513, and can scale up to 192, 336 and 528 ports, respectively. The ports for the Cisco MDS 9500 Series are added in different increments depending on whether you need 4 Gbps or 8 Gbps Fibre Channel. For 4 Gbps FC, the ports are added in 12-, 24- or 48-port increments. For 8 Gbps FC, the ports are added in 24- or 48-port increments, or a hybrid 4/44 model that has four 8 Gbps ports and 44 4 Gbps ports.

By contrast, regular Fibre Channel switches typically have a fixed physical port count, although blocks of ports may be enabled by a license key to keep the initial acquisition cost low. Port counts for these switches range from eight ports up to 64 or 80 ports for Brocade switches, and 128 ports for the QLogic 9000 in a single switch chassis.

Another advantage that directors provide is that with the large port count in a single unit, it may be possible to design a SAN fabric with minimal or no inter-switch links (ISLs). ISLs, sometimes known as "hops," contribute to increased latency and have the potential to degrade performance if they're not configured optimally. Often, smaller fixed-port count switches are deployed to get a SAN started. As storage growth occurs and the single switch is outgrown, a second switch is added and ISLs are used between the switches to expand the SAN fabric. As growth continues, a third switch is typically added and so on. At some point in this growth pattern with many smaller switches and several ports on each switch dedicated to ISLs, many of the ports won't be available to use as device ports (to connect host servers and storage units), so several hops may be necessary to connect a host server at one end of the SAN to a storage device at the other end of the SAN. Directors can alleviate this situation by removing many or all of the ISLs so that most of the switch ports can be used for devices rather than dedicating ports to the switching infrastructure.

Directors deliver more bandwidth

Bandwidth is another key advantage of director-class switches for environments that need high total I/O bandwidth. The Brocade DCX directors provide up to 256 Gbps of total bandwidth per slot (or blade) of 16, 32 or 48 ports. The DCX also has a local switching capability that can provide up to 384 Gbps of bandwidth for ports in the same port group within a 48-port blade. The Cisco 9500 series provides 96 Gbps or 192 Gbps per slot (or switching module) of 24 or 48 ports. Each vendor can provide performance data for their product that shows how well their systems do under heavy loads, but the main point is that they can handle very heavy loads of storage I/O traffic. In addition, these directors also have the capability to have several 10 Gbps Fibre Channel ports that are used for ISLs to trunk traffic between switches or directors. This ISL capability allows large SAN fabrics to be constructed in a core-edge type of topology, where one or more directors are at the core of the SAN and smaller, fixed port-count switches are deployed at the edge of the SAN. As server virtualization becomes more common and the load on physical servers becomes greater, the need for high bandwidth in the FC infrastructure also increases and directors become a more appealing alternative to regular switches.

High-availability benefits of directors

High availability is another design feature of directors. Normally, directors are deployed in business-critical settings where very high uptime is one of the key criteria of service-level agreements. Directors typically have redundant components so that there's no single point of failure. This covers the basics such as redundant hot-pluggable power supplies and cooling fans, and includes redundant core processing and switching components, as well as redundant World Wide Name (WWN) cards. These high-availability features allow for nondisruptive software or firmware upgrades.

Some of the redundant components operate in an active/passive mode, where one component is active and the other is in standby mode, ready to take over in the event of a failure of the active component. If there's a hardware failure on a port or port card in a director, the port card can be removed and replaced without losing use of the entire director. With an equivalent failure in a fixed port-count switch, the entire switch typically has to be taken offline for repair or replacement. Directors are designed to stay up and running so that host servers don't lose access to their storage data. Of course, all of the high-availability capabilities add to the cost of these systems.

High-end features of directors

Directors also provide a number of advanced features, some of which aren't available in smaller, fixed port-count switches. Directors from Brocade and Cisco each support the concept of virtual SANs, FICON for mainframes, and provide features for long-distance connections using FCIP, iSCSI, dense wavelength division multiplexing (DWDM), SONET and other protocols. The specific implementations differ slightly when comparing the two vendors' products, but these directors are designed to be the central hub for a complex SAN that not only handles local FC traffic, but can send the Fibre Channel traffic over distances to remote sites.

Because of their modular design, directors can adapt to new requirements. For example, specialty blades that support connecting to dark fiber, hardware encryption, 10 Gigabit Ethernet (10 GbE), Fibre Channel over Ethernet (FCoE) and other features are available today, and the directors' designs also support future requirements in the same infrastructure without having to replace the entire unit.

Management of directors and the SAN fabric is critical, and these directors provide solid management software that can do performance monitoring, quality of service (QoS) , bottleneck detection, advanced zoning and a myriad of other functions. The directors can also pass management data using SNMP to higher-level management software platforms.

Advanced capabilities equal higher cost

Fibre Channel switches and directors generally aren't purchased directly from the manufacturer but through a server vendor, storage vendor, system integrator or VAR. Pricing will vary depending on a number of factors, including your relationship with your supplier. Director pricing can get expensive, not only for the hardware but for some of the license keys. Per-port costs for fully loaded directors are higher than same port count fully loaded fixed-port switches, in the neighborhood of 2:1 or more, but you can scale up better with directors; directors also have more high-availability features built in.

For some data centers, available power is an issue as their electric power utility companies have indicated that they can't bring in any additional power to handle peak workloads. In these cases, data centers are clamoring for more energy-efficient equipment. Like most IT equipment vendors, switch and director vendors are focusing on energy efficiency. Given the ability of directors to consolidate the workloads of several or more regular switches, the case can be made that a move to directors will yield power conservation benefits, so you can expect director vendors to emphasize their "green" features.

Do you need a director? Before replacing your current switching infrastructure with a director, you need to determine your requirements. If you need any of the following, then you should consider a director-class switch:

• High availability in each component of the Fibre Channel infrastructure
• High port count
• High total bandwidth
• Lower percentage of ports dedicated to ISLs

You'll have to do some technical analysis to determine your specific needs, but if any of your requirements point to one or more of the above, then a director might be good solution.

The trade-offs between using directors and smaller, fixed port-count switches affect your equipment budget and operating budget. Directors are more expensive, but provide more features, as described previously. But you also need to consider the day-to-day management of your Fibre Channel infrastructure. Are you spending too much time managing too many "things"? Would it be easier to manage one large unit rather than several small units?

You should also consider if your storage infrastructure is in a growth mode, and if you'll need to add quite a few more ports now and in the near future -- and all in the same SAN. If so, a director's ability to add ports fairly easily might be a good option. If the additional ports you need were configured with smaller, fixed port-count switches, you should also account for the ports that would have to be dedicated to ISLs rather than to storage devices and servers.

As noted, if you do opt for one or more directors, you may still be able to use the smaller fixed-port switches that the directors are replacing. You could use those small switches as edge switches placed close to the servers or storage systems in a simple core-edge topology. You could also use the small switches to create a standalone testing environment.

Generally speaking, there are two good reasons to deploy switches and directors from the same manufacturer. First, the manufacturers are more likely to guarantee that the equipment is fully compatible with their own equipment. Secondly, you've probably already invested time, experience and money in training and getting comfortable with one brand of switches, so it will be easier to adapt to directors that have a similar look and feel rather than going through the time and expense of learning new equipment.

Directors can be very effective additions to your SAN, especially as you build for the future, but you'll have to carefully examine the features and pricing to ensure that they're the best solution for your shop.

About the author:
Dennis Martin has been working in the IT industry since 1980, and is the founder and president of Demartek, a computer industry analyst organization and testing lab.