Design pipeline using classification and embedding services
Problem Statement
Design an end-to-end system that processes user-uploaded text documents using two external, black-box Machine Learning (ML) services: a Classification Service and an Embedding Service.
Given ML Services
- Classification Service
- Input: One or more text documents.
- Output: A categorical label for each document (e.g., topic, category).
- Embedding Service
- Input: One or more text documents.
- Output: A fixed-dimension vector embedding (e.g., 768-dimensional float vector) for each document.
System Requirements
- File Uploads: Accept file uploads from users, where each file can contain one or more text documents.
- Upload Modes: Support both single-file uploads and bulk uploads of up to 1,000 files in a single request.
- Document Processing: For every document within an uploaded file:
- Compute a classification label using the Classification Service.
- Compute a vector embedding using the Embedding Service.
- Result Storage & Query: Store all processed results (raw files, parsed text, labels, embeddings) such that they can be queried later by user, file identifier, or via semantic search.
- Performance Goals:
- Achieve low latency for small, single-file uploads.
- Maintain high throughput for large, bulk uploads.
Task
Design the complete end-to-end pipeline and its associated APIs. Your design should specifically address the following areas:
- API Design
- Detail how clients will upload files, distinguishing between single and bulk uploads (up to 1,000 files).
- Specify the types of responses clients will receive (e.g., synchronous vs. asynchronous).
- Architecture
- Describe the orchestration mechanism for calling the Classification and Embedding services.
- Explain your strategy for storing raw files, parsed text, classification labels, and vector embeddings.
- Elaborate on how your architecture achieves both low latency for single uploads and high throughput for bulk uploads.
- Scalability & Performance
- Outline strategies to handle bulk uploads of 1,000 files without encountering memory exhaustion or violating latency targets.
- Discuss your approach to batching, queuing, and concurrency when interacting with the external ML services.
- Reliability & Observability
- Address error handling for partial failures (e.g., when some documents or files fail to process).
- Describe your plans for monitoring, logging, and metrics to ensure system health and debug issues.
Assumption
You cannot modify the internal implementation of the Classification and Embedding services; you may only interact with them via their exposed APIs.
AI interview prep
Core dilemma
Balancing the conflicting requirements of low-latency, synchronous processing for individual document uploads with high-throughput, asynchronous processing for bulk uploads, while efficiently managing calls to external, black-box ML services and ensuring robust error handling and data consistency across potentially large datasets.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design pipeline using classification and embedding services
Problem Statement
Design an end-to-end system that processes user-uploaded text documents using two external, black-box Machine Learning (ML) services: a Classification Service and an Embedding Service.
Given ML Services
- Classification Service
- Input: One or more text documents.
- Output: A categorical label for each document (e.g., topic, category).
- Embedding Service
- Input: One or more text documents.
- Output: A fixed-dimension vector embedding (e.g., 768-dimensional float vector) for each document.
System Requirements
- File Uploads: Accept file uploads from users, where each file can contain one or more text documents.
- Upload Modes: Support both single-file uploads and bulk uploads of up to 1,000 files in a single request.
- Document Processing: For every document within an uploaded file:
- Compute a classification label using the Classification Service.
- Compute a vector embedding using the Embedding Service.
- Result Storage & Query: Store all processed results (raw files, parsed text, labels, embeddings) such that they can be queried later by user, file identifier, or via semantic search.
- Performance Goals:
- Achieve low latency for small, single-file uploads.
- Maintain high throughput for large, bulk uploads.
Task
Design the complete end-to-end pipeline and its associated APIs. Your design should specifically address the following areas:
- API Design
- Detail how clients will upload files, distinguishing between single and bulk uploads (up to 1,000 files).
- Specify the types of responses clients will receive (e.g., synchronous vs. asynchronous).
- Architecture
- Describe the orchestration mechanism for calling the Classification and Embedding services.
- Explain your strategy for storing raw files, parsed text, classification labels, and vector embeddings.
- Elaborate on how your architecture achieves both low latency for single uploads and high throughput for bulk uploads.
- Scalability & Performance
- Outline strategies to handle bulk uploads of 1,000 files without encountering memory exhaustion or violating latency targets.
- Discuss your approach to batching, queuing, and concurrency when interacting with the external ML services.
- Reliability & Observability
- Address error handling for partial failures (e.g., when some documents or files fail to process).
- Describe your plans for monitoring, logging, and metrics to ensure system health and debug issues.
Assumption
You cannot modify the internal implementation of the Classification and Embedding services; you may only interact with them via their exposed APIs.