From Activity to Intent: Generating User Journeys with LLMs

This post details the transition from a legacy keyword clustering system to a generative LLM-based approach for user journey inference. It describes the prompt engineering for the LLM, including input formatting, action collapsing, and output constraints. It also covers the distillation process of a frontier teacher model into a 4B parameter student using SFT and LoRA, and the serving optimizations for batch processing at scale, including PagedAttention and continuous batching.