Context Assembly: Building the Prompt the Model Sees

This post elaborates on the concept of context assembly in LLM applications, detailing the components that form a prompt (system instructions, retrieved knowledge, conversation history, long-term memory, tool definitions) and the challenges associated with managing them within token limits. It introduces Redis Iris as a solution for low-latency context assembly, highlighting its capabilities in providing real-time context through Redis Agent Memory and semantic caching with Redis LangCache. The post also discusses the impact of input ordering on model performance and cost, and the failure modes associated with context management.