TechPrufer
Sign inGet started

Blogs›Together AI›Autonomous Data Scientist Agent

Autonomous Data Scientist Agent

Autonomous Data Scientist Agent

6
posts
2025–2026

Together AI enables the creation of autonomous data scientist agents by integrating open-source LLMs with a secure code execution environment (Together Code Interpreter - TCI). These agents leverage the ReAct (Reasoning and Action) pattern, specifically the CodeAct variant, to perform multi-step data analysis tasks. The system allows for flexible prompt engineering to tune agent behavior and supports dynamic installation of necessary libraries within the sandboxed execution environment. This post introduces DSGym, a unified framework that integrates diverse data science evaluation suites behind a single API with standardized abstractions for datasets, agents, and metrics. DSGym unifies and refines existing benchmarks while expanding the scope with novel scientific analysis tasks (90 bioinformatics tasks from academic literature) and challenging end-to-end modeling competitions (92 Kaggle competitions). Beyond evaluation, DSGym provides trajectory generation and synthetic query pipelines for agent training—we demonstrate this by training a 4B model on 2k generated examples, achieving state-of-the-art performance among open-source models. The framework abstracts code execution behind containers, comes with pre-installed dependencies, and offers a unified JSON interface for all benchmarks. DSGym categorizes tasks into Data Analysis and Data Prediction, and introduces DSBio and DSPredict datasets. It also includes a data generation pipeline for agent training, producing 2,000 high-quality query-trajectory pairs for supervised finetuning. The post addresses the 'memorization gap' by filtering out tasks solvable without data access and presents benchmark performance and failure modes, showing that even frontier models underperform on multi-step reasoning, with scientific analysis tasks dominated by domain-grounding errors. DSPredict evaluates the ability of agents to build complete machine learning pipelines, with performance measured by valid submission, median/percentile, and medal achievements.

2026

EinsteinArena: Harnessing the collective intelligence of agents in the wild to advance science

4/13/2026

This post introduces EinsteinArena, a platform for AI agents to collaborate on scientific discovery, building upon the concept of autonomous agents. It details how agents can interact, share ideas, and collectively solve complex problems, exemplified by achieving new state-of-the-art results on mathematical problems like the Kissing Number problem. The platform features a live API and leaderboard system, real-time discussion threads for collaboration, and a robust verifier design for trustworthy results. The open-sourcing of the platform is also highlighted.

DSGym: A holistic framework for evaluating and training data science agents

1/26/2026

Introduces DSGym, a unified framework for evaluating and training data science agents. DSGym integrates diverse data science evaluation suites behind a single API with standardized abstractions for datasets, agents, and metrics. It expands the scope with novel scientific analysis tasks (90 bioinformatics tasks) and challenging end-to-end modeling competitions (92 Kaggle competitions). DSGym provides trajectory generation and synthetic query pipelines for agent training, demonstrating training a 4B model on 2k generated examples. The framework abstracts code execution behind containers, offers a unified JSON interface, and categorizes tasks into Data Analysis and Data Prediction. New datasets DSBio and DSPredict are introduced. Addresses the 'memorization gap' by filtering tasks solvable without data access and analyzes benchmark performance and failure modes, highlighting domain-grounding errors in scientific tasks. Evaluates agent performance on building ML pipelines via DSPredict.

2025

How Together AI Uses AI Agents to Automate Complex Engineering Tasks: Lessons from Developing Efficient LLM Inference Systems

8/21/2025

This post details six patterns for building AI agents to automate complex engineering tasks: Infrastructure Patterns (Good Tools, Documentation, Safe Execution) and Behavioral Patterns (Manage Parallel Sessions, Manage Wait Time, Progress Monitoring). It provides a case study on using these patterns to automate the training pipeline for speculator models, a key technique for accelerating LLM inference. The post highlights how these agents reduce manual intervention, improve consistency and reliability, and decrease turnaround time for developing efficient LLM inference systems.

Back to The Future: Evaluating AI Agents on Predicting Future Events

7/17/2025

This post introduces FutureBench, a new benchmark for evaluating AI agents' ability to predict future events. It details the methodology for generating prediction questions from news and prediction markets, the technical stack used (DeepSeek-V3, Firecrawl, Tavily), and a three-level evaluation framework (framework comparison, tool performance, model capabilities). The post also presents initial results comparing different LLMs (GPT-4.1, Claude3.7, DeepSeek-V3) and analyzes their action patterns and prediction strategies.

DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL

7/2/2025

Introduces DeepSWE-Preview, a state-of-the-art open-source coding agent trained from Qwen3-32B using only reinforcement learning (RL). Details the training recipe, including scalable dataset curation with R2E-Gym, environment setup with tools like Bash execution and file editing, and a sparse outcome reward model. The training process utilizes an enhanced GRPO++ algorithm with features like Clip High, No KL Loss, and Compact Filtering for stable and performant training of multi-turn agents. Integrates Kubernetes for scalable agent rollout collection, managing thousands of containers for RL training.

From Zero to One: Building An Autonomous and Open Data Scientist Agent from Scratch

6/12/2025

This post details the architecture and implementation of an autonomous data scientist agent built from scratch. It highlights the use of the ReAct pattern (specifically CodeAct) for agentic reasoning and action, and introduces the Together Code Interpreter (TCI) as a secure and efficient environment for executing Python code generated by the agent. The post emphasizes the modularity gained by decoupling reasoning logic from code execution and showcases practical examples of the agent's capabilities, including self-correction and automated report generation.