TechPrufer
Get started
Dropbox
Dropbox logoDropbox · Feature trail

400G Networking Infrastructure

3 posts · Oct 2018 – Nov 2023

What this trail covers

Dropbox has designed and launched its first data center architecture using 400 gigabit per second (400G) ethernet technology. This upgrade significantly increases bandwidth, reduces power usage and cabling footprint, and streamlines data center interconnects. The new design leverages advanced technologies like digital signal processing for higher data rates and uses 400G direct attach copper (DAC) cabling for energy-efficient spine-leaf interconnections. The top-of-rack interconnects utilize 400G.

All Dropbox trails
  1. From AI to sustainability, why our latest data centers use 400G networkingThis post details the design and implementation of Dropbox's first data center architecture utilizing 400G ethernet. It covers the rationale for adopting 400G, including meeting the bandwidth requirements of AI workloads and sustainability goals. Key technical aspects discussed include the use of 400G DAC cabling for fabric core interconnections, achieving a 3x energy efficiency improvement per Gigabit. It also describes the selection of 400G-DR4 optics for top-of-rack interconnects to support both 100G and 400G switches up to 500 meters. Furthermore, the post explains the consolidation of the data center interconnect layer from three devices into one, enabled by 400G technology.
  2. Boosting Dropbox upload speed and improving Windows’ TCP stackThis post details a collaborative effort with Microsoft to diagnose and resolve slow upload speeds on Windows clients. The issue was identified as packet reordering within the Windows TCP stack, exacerbated by the interaction with NIC firmware. The collaboration led to improvements in the Windows implementation of the TCP RACK-TLP algorithm, resulting in a fix released by Microsoft. The post also highlights the utility of Windows tracing tools like `netsh trace` and Message Analyzer compared to Linux-centric approaches.
  3. Dropbox traffic infrastructure: Edge networkThis post details Dropbox's Edge network infrastructure, including its points of presence (PoPs) around the world, Global Server Load Balancing (GSLB) strategies (BGP anycast and GeoDNS), and the rationale behind building their own Edge network to improve latency and file transfer performance. It discusses PoP location selection algorithms, the challenges and benefits of anycast, and the tools used for troubleshooting. The post also touches upon the evolution of their GSLB from anycast to GeoDNS for better traffic steering and graceful drain capabilities.