TechPrufer
Get started
Apple
Apple logoApple · Feature trail

Agentic Data Cataloging

4 posts · Feb 2025 – Oct 2026

What this trail covers

This post introduces RISED (Rubrics for Agentic Multi-Environment Selection and Self-Distillation), a novel framework for training LLM agents across diverse interactive environments. RISED repurposes rubrics, originally used for reward signals, to guide online data selection and policy supervision. An LLM judge tags rollouts with a predefined rubric vocabulary, which then guides the selection of data aligning with the overall behavioral composition of the mixed-environment batch. Positive rubrics provide privileged context for self-distillation, while negative rubrics guide generation away from failure modes. RISED achieves state-of-the-art performance across environments and offers rubric-based analysis of behavioral changes.

All Apple trails
  1. RISED: Rubrics for Agentic Multi-Environment Selection and Self-DistillationRISED introduces a rubric-based approach for agentic multi-environment training. It uses an LLM judge to tag rollouts with a shared rubric vocabulary, enabling data selection based on behavioral composition and limiting overlap. Positive rubrics facilitate self-distillation for token-level supervision, and negative rubrics help avoid failure modes. This method improves agent performance and provides behavioral insights.
  2. SCLATE: A Substrate for Continual-Learning Agent Training and EvaluationIntroduces SCLATE, an execution substrate for continual-learning agent training and evaluation. SCLATE features an open event scheduler, a hybrid simulated clock for time compression, and a rollout engine that records model calls. The post presents empirical comparisons of agent harness and memory configurations, and details the post-training of Qwen3.5-4B using SCLATE, showing improvements in SWE-bench Verified pass rate and MetaClaw accuracy.
  3. Glyph: A Multi-Strategy Agentic System for Column Description and Sensitivity-Ontology Tagging of Enterprise Data CatalogsThis post introduces Glyph, a production system that frames column description generation and data classification tagging as cooperating LLM agents. It utilizes a Descriptor agent with active RAG grounded in enterprise GitHub code via a reasoning-acting tool loop. The Tagger agent employs three parallel strategies (description, regex, metadata) and fuses their outputs with Reciprocal Rank Fusion (RRF). A fine-tuned MiniLM metadata encoder significantly improves retrieval accuracy. The system emphasizes value-free, code-grounded design, per-tag provenance, and graceful degradation for auditable and operable cataloging.
  4. Environment-free Synthetic Data Generation for API-Calling AgentsThis post proposes an environment-free synthetic data generation approach for training API-calling LLM agents. It leverages LLMs as on-the-fly digital world models to generate diverse tasks, simulate coherent API responses, and filter trajectories, overcoming the bottleneck of requiring fully implemented environments. This method demonstrates significant performance gains on benchmarks like AppWorld and OfficeBench when fine-tuning models on the generated synthetic data.