Sign in to track progress, unlock prep layers, and browse your pipeline.Sign in
Design CSV upload endpoint with GPT classification
Design CSV Upload Endpoint with GPT Classification
Problem Description
You are tasked with designing and implementing a backend service. This service must process two uploaded CSV files and then utilize an external GPT-like API for classification.
Requirements
1. HTTP Endpoint
- Expose an HTTP endpoint, for example,
POST /ingest-data. - The client will upload two distinct CSV files within a single request:
users.csv: A typical row might beuser_id,name,email.tasks.csv: A typical row might betask_id,user_id,description.
2. CSV Parsing and Local JSON Storage
- The endpoint should:
- Receive the two CSV files.
- Parse their contents into in-memory data structures (e.g., lists of objects).
- Serialize each parsed dataset into JSON format.
- Persist the resulting JSON data to the local filesystem (e.g.,
users.json,tasks.json).
3. GPT Classification Step
- After parsing and local storage, the service must call an external GPT-like API to classify a specific field within the JSON data.
- For example, for each task in
tasks.json, classify thedescriptionfield into one of a small, predefined set of categories (e.g., "bug", "feature", "documentation").
- For example, for each task in
- The GPT API characteristics:
- Accessed via HTTPS.
- Accepts a text prompt and returns a classification label in JSON format.
- You have the flexibility to design the prompt and decide whether to call the GPT API per-record or in batches, ensuring all tasks receive a classification label.
4. Response
- Upon completion of the classification, the service should return an HTTP response that includes at least:
- A clear success indicator.
- Basic statistics (e.g., number of users processed, number of tasks processed).
- Optionally, the enriched
tasksdata, now including the new classification field.
5. Non-functional Requirements
- Implement robust handling for basic validation and error cases (e.g., missing files, malformed CSV data, failures from the GPT API).
- The solution must gracefully handle multiple clients concurrently calling this endpoint.
- The overall design should be reasonably testable.
Task
Describe your design and implementation approach for this endpoint. Your description should cover:
- The HTTP API contract (request and response formats).
- Strategies for handling file uploads and CSV parsing.
- How you structure the code for writing JSON to local storage.
- Integration with the GPT classification API, including error handling and considerations for batching.
- Key considerations for concurrency, timeouts, and testing.
AI interview prep
Core dilemma
Balancing synchronous, potentially long-running external API calls (GPT classification) and local I/O (file storage) within an HTTP request-response cycle, while maintaining responsiveness, scalability, and handling concurrency and failures.
Mid-level signals3
Senior signals4
Red flags4
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design CSV upload endpoint with GPT classification
Design CSV Upload Endpoint with GPT Classification
Problem Description
You are tasked with designing and implementing a backend service. This service must process two uploaded CSV files and then utilize an external GPT-like API for classification.
Requirements
1. HTTP Endpoint
- Expose an HTTP endpoint, for example,
POST /ingest-data. - The client will upload two distinct CSV files within a single request:
users.csv: A typical row might beuser_id,name,email.tasks.csv: A typical row might betask_id,user_id,description.
2. CSV Parsing and Local JSON Storage
- The endpoint should:
- Receive the two CSV files.
- Parse their contents into in-memory data structures (e.g., lists of objects).
- Serialize each parsed dataset into JSON format.
- Persist the resulting JSON data to the local filesystem (e.g.,
users.json,tasks.json).
3. GPT Classification Step
- After parsing and local storage, the service must call an external GPT-like API to classify a specific field within the JSON data.
- For example, for each task in
tasks.json, classify thedescriptionfield into one of a small, predefined set of categories (e.g., "bug", "feature", "documentation").
- For example, for each task in
- The GPT API characteristics:
- Accessed via HTTPS.
- Accepts a text prompt and returns a classification label in JSON format.
- You have the flexibility to design the prompt and decide whether to call the GPT API per-record or in batches, ensuring all tasks receive a classification label.
4. Response
- Upon completion of the classification, the service should return an HTTP response that includes at least:
- A clear success indicator.
- Basic statistics (e.g., number of users processed, number of tasks processed).
- Optionally, the enriched
tasksdata, now including the new classification field.
5. Non-functional Requirements
- Implement robust handling for basic validation and error cases (e.g., missing files, malformed CSV data, failures from the GPT API).
- The solution must gracefully handle multiple clients concurrently calling this endpoint.
- The overall design should be reasonably testable.
Task
Describe your design and implementation approach for this endpoint. Your description should cover:
- The HTTP API contract (request and response formats).
- Strategies for handling file uploads and CSV parsing.
- How you structure the code for writing JSON to local storage.
- Integration with the GPT classification API, including error handling and considerations for batching.
- Key considerations for concurrency, timeouts, and testing.