DSGym: A holistic framework for evaluating and training data science agents

Introduces DSGym, a unified framework for evaluating and training data science agents. DSGym standardizes data science benchmarks with a single API and abstractions for datasets, agents, and metrics. It expands benchmark scope with new bioinformatics (DSBio) and Kaggle competition (DSPredict) datasets. The framework includes a data generation pipeline for agent training, demonstrating its effectiveness by training a 4B model on 2,000 generated examples to achieve state-of-the-art performance. The post also highlights failure modes in current LLMs for data science tasks, particularly domain-grounding errors in scientific analysis.