Jev and the return of the zero-shot classifier

This post introduces Jev, a "System One" model designed for typed decisions (classification, scoring, yes-or-no answers) that offers zero-shot flexibility via a hosted API. It details Jev's advantages over general-purpose LLMs, open zero-shot classifiers, and fine-tuned classifiers by comparing tradeoffs in latency, cost, flexibility, and quality. Experiments at Glean demonstrate Jev's effectiveness in query classification (achieving 66.8% agreement against LLM baselines, with potential for significant speedup) and model routing (outperforming a traditional LLM-based router in both accuracy and latency for a simplified 3-expert scenario). It also shows promise in reranking search results, though specialized models still outperform it on certain classification tasks. The post highlights the importance of Jev for tasks where the taxonomy changes frequently, avoiding the need for retraining.