Build and design a Mistral RAG agent
Design and Implement a Minimal LLM-Powered RAG Agent
Context
You are tasked with designing and implementing a minimal, yet production-minded, Retrieval-Augmented Generation (RAG) agent in Python. This agent will leverage the Mistral API for both embeddings and chat completions. It should operate over a small local corpus, such as Markdown and PDF files, support streaming responses, maintain a short conversation memory, and incorporate basic reliability features.
Assume the use of Python 3.10+ and that external dependencies can be installed. The Mistral API token will be provided via an environment variable.
Deliverables
Implement a minimal Python tool that fulfills the following requirements:
- Corpus Ingestion: Ingest a small local corpus (e.g., Markdown and PDF files), chunk the text, generate embeddings, and build a searchable index.
- Information Retrieval: Retrieve the top-k most relevant passages for a given user query.
- LLM Interaction: Compose a prompt and invoke the Mistral chat/completions API with support for streaming responses.
- Conversation Management: Maintain short-term conversation memory to provide context for subsequent queries.
- Configuration: Read the Mistral API token securely from an environment variable.
- Reliability: Implement error handling, retry mechanisms, and rate limit management for API calls.
- Documentation & Testing: Include a brief README file explaining the setup and usage, along with basic smoke tests to verify functionality.
Additionally, be prepared to explain your system design choices, covering the following aspects:
- Chunking strategy
- Embedding model choice
- Vector index selection
- Latency and cost budgeting
- Caching mechanisms
- Prompt templating strategies
- Safety and PII handling
- Logging and monitoring
- Offline RAG evaluation
- Fallbacks when retrieval fails
AI interview prep
Core dilemma
Balancing the 'minimal' implementation requirement with 'production-minded' robustness, efficiency, and quality for a RAG system, demonstrating awareness of scalability and reliability implications even at a small scale.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Build and design a Mistral RAG agent
Design and Implement a Minimal LLM-Powered RAG Agent
Context
You are tasked with designing and implementing a minimal, yet production-minded, Retrieval-Augmented Generation (RAG) agent in Python. This agent will leverage the Mistral API for both embeddings and chat completions. It should operate over a small local corpus, such as Markdown and PDF files, support streaming responses, maintain a short conversation memory, and incorporate basic reliability features.
Assume the use of Python 3.10+ and that external dependencies can be installed. The Mistral API token will be provided via an environment variable.
Deliverables
Implement a minimal Python tool that fulfills the following requirements:
- Corpus Ingestion: Ingest a small local corpus (e.g., Markdown and PDF files), chunk the text, generate embeddings, and build a searchable index.
- Information Retrieval: Retrieve the top-k most relevant passages for a given user query.
- LLM Interaction: Compose a prompt and invoke the Mistral chat/completions API with support for streaming responses.
- Conversation Management: Maintain short-term conversation memory to provide context for subsequent queries.
- Configuration: Read the Mistral API token securely from an environment variable.
- Reliability: Implement error handling, retry mechanisms, and rate limit management for API calls.
- Documentation & Testing: Include a brief README file explaining the setup and usage, along with basic smoke tests to verify functionality.
Additionally, be prepared to explain your system design choices, covering the following aspects:
- Chunking strategy
- Embedding model choice
- Vector index selection
- Latency and cost budgeting
- Caching mechanisms
- Prompt templating strategies
- Safety and PII handling
- Logging and monitoring
- Offline RAG evaluation
- Fallbacks when retrieval fails