Internet Latency and Network Performance Analysis
Opening designs for 6-pack and Wedge 100

Opening designs for 6-pack and Wedge 100

3/9/2016 · Zhiping Yao, Hany Morsy, Jasmeet Bagga

What this post added

This post details the opening of designs for the 6-pack and Wedge 100 modular switches, representing a significant step towards supporting 100G connectivity in Meta's data centers. It describes the hardware specifications of Wedge 100 (a 32x100G TOR switch) and 6-pack (a 128x40G modular switch), including their use of Broadcom's Tomahawk ASIC, support for OpenRack v2, COM-E CPU modules, and various QSFP28 and QSFP+ optic transceivers and DAC cables. The post also elaborates on the extensive hardware and network-level testing performed for these switches within the data center fabric, highlighting the increased scalability and performance requirements for fabric switches compared to top-of-rack switches (e.g., 16K routes vs. 1K, 5.12Tbps traffic rate vs. 640Gbps). It details critical test scenarios for functionality (BGP peerings, route policy processing, ECMP) and failure conditions (BGP flapping, link failures, card insertions/removals), demonstrating sub-2-millisecond failure handling and rerouting with minimal packet loss. Furthermore, the post discusses the software architecture of FBOSS running on 6-pack, emphasizing a disaggregated approach where the chassis switch is viewed as a set of connected Wedges rather than a traditional supervisor module model, and highlights the ongoing effort to automate the entire test suite.

Read the original post ↗