Together Evaluations now supports comparing top commercial APIs vs. open source models
Feb 2, 2026
Summary
This post announces the integration of proprietary models (OpenAI, Anthropic, Google) as both judges and target models within the Together Evaluations framework. It also adds support for evaluating Together fine-tuned models (LoRA serverless inference, Dedicated Endpoints). New recipes are provided for fine-tuning open-source judges to outperform proprietary ones (e.g., GPT-5.2) and for using GEPA to automate prompt optimization. The core functionality of uploading data, picking evaluation types (classify, score, compare), describing evaluation criteria via system templates, and configuring models (external with API key, or fine-tuned model IDs) remains the same.
More from the LLM Evaluation Framework trail
- What do LLMs think when you don't tell them what to think about?
- Fine-tuning open LLM judges to outperform GPT-5.2
- How to choose the right open model for production
- How to evaluate and benchmark Large Language Models (LLMs)
- Dynamic AI agent testing for the real world with Collinear Simulations and Together Evals