Flow generation through natural language: An agentic modeling approach (2026) - Shopify

What this post added

This post details the fine-tuning of a tool-calling agent to generate Shopify Flows from natural language for Sidekick, Shopify's AI commerce assistant. The approach involves creating a training dataset by reverse-engineering existing production workflows, generating natural language queries, and constructing tool trajectories. To improve model performance, workflows are represented in Python as an intermediate DSL, which is then transpiled to Shopify's native JSON DSL. The training data closely mirrors the production environment, including tool naming, ordering, response formats, and system prompts. The tool-calling stack is optimized by returning lightweight summaries first, followed by full details only when necessary. Training is accelerated using H200 GPUs and Shopify's Tangle ML experimentation platform. Initial production deployment revealed that offline benchmarks masked a significant gap in handling out-of-distribution requests, highlighting the importance of real-world user feedback and continuous learning.