TechPrufer
Get started
Hudson River Trading
Hudson River Trading logoHudson River Trading · Feature trail

Blobby Distributed Filesystem

2 posts · Dec 2021 – Aug 2025

What this trail covers

HRT built and operates Blobby, a custom distributed filesystem designed to meet its specific needs for storing and analyzing vast amounts of data. Blobby addresses limitations in cost, scale, scalability, multi-tenancy, and control found in existing solutions. It employs a design based on prior art, focusing on high aggregate throughput, durability, and simplicity. Key architectural components include client libraries, chunk servers, a FoundationDB-backed metadata database, proxy servers, and background services like Shepherd and Repair. The system supports traditional replication and erasure coding for data durability, and utilizes FoundationDB for ACID-compliant, horizontally scalable metadata management. This post details the architecture, including data layout, client API, data placement strategies using Copysets, chunk server implementation, metadata DB choices, proxy server roles, and background services for health monitoring and recovery.

All Hudson River Trading trails
  1. Building a Distributed Filesystem for Scalable ResearchThis post details the architecture of Blobby, HRT's custom distributed filesystem. It covers design principles (high aggregate throughput, durability, simplicity), data layout (replication and erasure coding with 6+3 default), client API (C++ library for metadata, read, write operations), data placement (Copysets and rack awareness), chunk server implementation (storing chunks as XFS files), metadata DB choice (FoundationDB for scalability, fault tolerance, ACID transactions, and strict serializability), proxy server roles (handling non-data operations and write steering), and background services (Shepherd for health checks and Repair for data recovery).
  2. Enabling Algo Research on Blockchain DataThis post details the process of structuring and querying blockchain data, specifically Ethereum transactions and internal transactions, using a distributed query engine (Trino) and a custom Geth node setup to overcome limitations in public blockchain data accessibility and analysis. It describes the challenges of accessing and processing 'internal transactions' which are not directly available from standard Ethereum nodes, and how the `debug_traceTransaction` API of Geth was leveraged. The post also covers the setup of a Geth node with full sync to obtain historical trace data and the parsing of EVM execution traces to extract detailed transaction information.