How to optimize token efficiency in agentic systems

This post details how to optimize token efficiency in agentic systems by shifting focus from prompt editing to workflow design. It identifies token waste in retrieval noise, bloated working context, unstructured memory, over-orchestration, and unbounded loops. The proposed solutions include measuring token usage by workflow stage, reducing context before it reaches the model via progressive retrieval and selective passing of evidence, replacing raw transcripts with structured state (working state, session summary, long-term facts), routing tasks by complexity, designing loops with clear stopping rules, and reusing proven plans and stable context. Key metrics to track include tokens per successful task, cost per completed workflow, latency per workflow, retrieval precision, groundedness, human correction rate, retry frequency, and context utilization rate.