AI Assistant Routing and Context Management

1 post · Aug 2026

What this trail covers

Grab's AI Assistant for account managers, Jarvis Pro, has evolved with a 'route first, answer later' design. This system classifies user prompts to determine the appropriate task type, scope, and necessary context before generating an answer. This approach aims to prevent common errors by ensuring the system understands the user's intent and loads only relevant information, improving the accuracy and safety of AI-generated recommendations. The post details the router's design, memory management, and evaluation strategies, emphasizing the tradeoff between first token latency and response safety. Regression checks were implemented to reconcile metric discrepancies between reporting surfaces.

All Grab trails

Aug 2026Building Jarvis Pro: Route first, answer laterThis post details the evolution of Jarvis Pro's architecture with a 'route first, answer later' design. Key technical contributions include the development of a prompted LLM classification router that determines task type, scope, and necessary context, preventing common errors by loading only relevant information. The system's memory management was refined to keep context narrow and prioritize explicit context over saved defaults. A deliberate tradeoff was made between first token latency and response safety, with slower first tokens being acceptable for safer recommendations. Regression checks were implemented to normalize source values and compare daily rows across approved metric paths for metric reconciliation. Evaluation was conducted using a routing set and an answer-quality suite, focusing on accepted route accuracy and rubric-based scoring.
Advertisement