//
you're reading...
Data Center

Data center Top of Rack and End of Row Design

Cabling architectures lay the infrastructure foundation for data center networks. Sometimes overlooked, physical layer connectivity is a necessity that requires careful planning and consideration when choosing a cabling architecture. There are several types of cabling architectures found in the data center; direct-connect, zone-distribution, and top-of-rack (ToR).

Each architecture offers advantages over the others depending on DC function, uptime tier level, growth (planned and unplanned), and equipment density. A “one-size-fits-all” approach to each of these architectures also may not fit everyone’s needs and thus hybrid approaches are commonly used.
This white paper seeks to give a detailed analysis of ToR cabling architectures. Advances made in the networking world, including the introduction of 10 Gigabit Ethernet (10GbE) and Fibre Channel over Ethernet (FCoE) to the server, have leveraged ToR architectures as a solution. Cabling architectures lay the infrastructure foundation for data center networks. Sometimes overlooked, physical layer connectivity is a necessity that requires careful planning and consideration when choosing a cabling architecture. There are several types of cabling architectures found in the data center; direct-connect, zone-distribution, and top-of-rack (ToR). Each architecture offers advantages over the others depending on DC function, uptime tier level, growth (planned and unplanned), and equipment density. A “one-size-fits-all” approach to each of these architectures also may not fit everyone’s needs and thus hybrid approaches are commonly used.This white paper seeks to give a detailed analysis of ToR cabling architectures. Advances made in the networking world, including the introduction of 10 Gigabit Ethernet (10GbE) and Fibre Channel over Ethernet (FCoE) to the server, have leveraged ToR architectures as a solution.

Summary of Top of Rack advantages (Pro’s):
Copper stays “In Rack”. No large copper cabling infrastructure required.
Lower cabling costs. Less infrastructure dedicated to cabling and patching. Cleaner cable management.
Modular and flexible “per rack” architecture. Easy “per rack” upgrades/changes.
Future proofed fiber infrastructure, sustaining transitions to 40G and 100G.
Short copper cabling to servers allows for low power, low cost 1oGE (10GBASE-CX1), 40G in the future.
Ready for Unified Fabric today.

Summary of Top of Rack disadvantages (Con’s):
More switches to manage. More ports required in the aggregation.
Potential scalability concerns (STP Logical ports, aggregation switch density).
More Layer 2 server-to-server traffic in the aggregation.
Racks connected at Layer 2. More STP instances to manage.
Unique control plane per 48-ports (per switch), higher skill set needed for switch replacement.
Data centers that are built in ToR networks use low port count switches (typically ≤48 ports) placed in each server cabinet. Designation for the switch to be placed at the top of the cabinet allows for easier management of intra-cabinet patch cords routed to each server; however implementations where the switch is placed at the bottom or middle of the cabinet are not uncommon.
Inter-cabinet cabling (cables run between server cabinets) for ToR architectures typically consists of a lower ratio of connections (fiber or copper) exiting the cabinet than are needed to support each server within the cabinet. This allows the ToR switch to act as a cabling consolidation point. Aggregation switches are used to feed each ToR switch and most times are located in a central physical area.

Top of Rack Architecture
ToR architectures are typically used when deploying dense server environments that require several network connections per server. This is commonly found in clustered environments, load balancing appliances, web search engines, or in virtualized environments that share processing of several separate physical servers located within a single cabinet.

Intra- and inter-cabinet (Figure 1) cabling is easier to manage as ToR switches, which aim to reduce connections, will see a cable management benefit with ToR architectures. Within the cabinet it is common to find a vertical bundle of patch cords that are dressed properly and fanned out to each server port. With non-ToR architectures, managing these cable bundles in a  centralized cross-connect or inter-connect area may present challenges.

End of Row

Advertisements

Discussion

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: