Design a Cloud File Storage and Sync Service (Google Drive)
Design a Cloud File Storage and Sync Service (Google Drive)
Design a cloud file storage and synchronization service similar to Google Drive or Dropbox. Users should be able to upload files of any type and size from web, desktop, and mobile applications. The service must allow access to these files from any device, keep a local folder synchronized with the cloud in near real-time, and enable sharing of files or folders with other users. The design should cover how bytes and metadata are stored, the upload/download paths, multi-device synchronization, and sharing/permissions, all while operating at scale and ensuring data durability.
Constraints & Assumptions:
- Users: Approximately 100 million registered users, with ~10 million daily active users.
- Storage: Average user stores ~50 GB. Individual files range from KB to several GB, with a single-file upload cap of 5 GB.
- Workload: Read-heavy, but with substantial write/upload volume. Uploads can be large and long-running.
- Synchronization: Desktop client should synchronize a watched folder in near real-time (changes appear on other online devices within seconds).
- Durability: Paramount; no data loss on single host, disk, or region failure. Target availability ~99.9%.
- Versioning: Files are mutable; support version history and restoration of previous versions.
- Access Patterns: Most access is to a user's own files; sharing constitutes a smaller fraction of traffic.
Key Design Areas:
- Storage and Upload/Download Path: Design how file bytes and metadata are stored. Detail upload/download flows, handling large files, content deduplication, and resuming interrupted uploads.
- Multi-Device Synchronization: Design how changes propagate between devices within seconds. Cover local change detection, learning about remote changes, and resolving conflicting edits.
- Sharing, Permissions, and Scale/Reliability: Extend the design to support sharing files/folders with specific users or via links, with view/edit permissions. Discuss scaling the metadata store and ensuring system durability and availability.
AI interview prep
Core dilemma
Balancing high durability and availability guarantees for potentially petabytes of user data against the complexity and cost of a highly distributed, fault-tolerant storage and synchronization system, while also meeting strict latency requirements for real-time sync.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design a Cloud File Storage and Sync Service (Google Drive)
Design a Cloud File Storage and Sync Service (Google Drive)
Design a cloud file storage and synchronization service similar to Google Drive or Dropbox. Users should be able to upload files of any type and size from web, desktop, and mobile applications. The service must allow access to these files from any device, keep a local folder synchronized with the cloud in near real-time, and enable sharing of files or folders with other users. The design should cover how bytes and metadata are stored, the upload/download paths, multi-device synchronization, and sharing/permissions, all while operating at scale and ensuring data durability.
Constraints & Assumptions:
- Users: Approximately 100 million registered users, with ~10 million daily active users.
- Storage: Average user stores ~50 GB. Individual files range from KB to several GB, with a single-file upload cap of 5 GB.
- Workload: Read-heavy, but with substantial write/upload volume. Uploads can be large and long-running.
- Synchronization: Desktop client should synchronize a watched folder in near real-time (changes appear on other online devices within seconds).
- Durability: Paramount; no data loss on single host, disk, or region failure. Target availability ~99.9%.
- Versioning: Files are mutable; support version history and restoration of previous versions.
- Access Patterns: Most access is to a user's own files; sharing constitutes a smaller fraction of traffic.
Key Design Areas:
- Storage and Upload/Download Path: Design how file bytes and metadata are stored. Detail upload/download flows, handling large files, content deduplication, and resuming interrupted uploads.
- Multi-Device Synchronization: Design how changes propagate between devices within seconds. Cover local change detection, learning about remote changes, and resolving conflicting edits.
- Sharing, Permissions, and Scale/Reliability: Extend the design to support sharing files/folders with specific users or via links, with view/edit permissions. Discuss scaling the metadata store and ensuring system durability and availability.