System Design Task: Minimal Conversational Chatbot

Goal

Design and implement a minimal yet functional conversational chatbot using the OpenAI API. This exercise is scoped for a two-hour coding session, with the expectation that the solution can be run locally.

Requirements

1. Architecture

The design must include:

• An API server component.
• A defined message schema for communication.
• A strategy for managing session state.
• A robust context window strategy to effectively manage conversation history within the OpenAI model's token limits.

2. Streaming Responses

The chatbot must support streaming responses back to the client.

3. Resilience

The system should be resilient, specifically addressing:

• Handling OpenAI API rate limits (HTTP 429 errors).
• Implementing retry mechanisms for transient failures.
• Graceful handling of generic API errors.

4. Operations

Operational considerations include:

• Implementing logging and metrics to facilitate quick debugging.
• Choosing and integrating a lightweight persistence option for relevant data.
• Incorporating basic safety checks, such as content moderation or guardrails, to ensure appropriate interactions.

Deliverables

Candidates are expected to provide:

• A clear project structure.
• Definitions for key API endpoints.
• Exact, reproducible steps to run the solution locally.
• Optional: A discussion on how an AI IDE (e.g., Cursor) could accelerate delivery for such a project.