TechPrufer
Get started
Hugging Face
Hugging Face logoHugging Face · Feature trail

CLI for AI Agents

16 posts · Apr 2024 – Sep 2026

What this trail covers

The Hugging Face CLI (`hf`) has been re-architected to optimize for both human and AI agent usage, including agent-specific output formats, next-command hints, non-blocking operations, and discoverable commands. The Agentic Resource Discovery (ARD) specification is introduced as a discovery layer for agents to find capabilities at runtime. Hugging Face's Discover Tool implements ARD, providing search access to Skills, ML applications, and MCP Servers. The Hub's existing semantic search is leveraged for agentic workflows. This post introduces ProvenanceGuard, a source-aware verification layer for MCP agents that ensures claims are attributed to the correct source, preventing cross-source conflation. It breaks down agent answers into claims, identifies relevant sources, verifies support, checks attribution, and can trigger a repair loop for blocked answers. The system uses local models for verification and can be adapted for cloud services.

All Hugging Face trails
  1. Give Your Coding Agents a Memory You OwnIntroduced `funes`, a durable memory layer for coding agents (Claude Code, Codex, pi, Hermes). `funes` indexes agent session traces locally using Lance datasets and can optionally publish to Hugging Face datasets for cross-agent and cross-machine memory sharing. It provides `recall` and `ask` tools for agents and users to retrieve specific session information, grounding responses in evidence. The system prioritizes local processing for privacy and performance, with optional cloud storage for persistence and collaboration. It addresses the problem of agents starting from zero by providing a mechanism to access and utilize past reasoning and decisions.
  2. Beyond LLMs: Why Scalable Enterprise AI Adoption Depends on Agent LogicThis post details how agent logic, implemented through software primitives like knowledge graphs and program analysis libraries, can steer LLMs within an agent harness to achieve more performant and cost-effective enterprise AI adoption. It presents four key domains where agent logic is applied: understanding legacy code (Cobol/PL/1) using program analysis and static analysis for IBM watsonx Code Assistant for Z; expediting test generation with the Aster library for Java applications; proactively responding to incidents and enabling shift-left app resiliency through observability-driven orchestration with knowledge graphs and program analysis; and automating IT compliance modernization using algorithms and adaptive planning. Case studies in healthcare (Configurable Generalist Agent - CUGA) and condition-based maintenance for physical assets further illustrate these principles.
  3. Model Routing Is Simple. Until It Isn’t.This post details the challenges and solutions for building an effective model routing system for AI agents, shifting from a classification problem to a systems optimization problem. It highlights that cost is more than model pricing due to caching, complexity is more than task difficulty due to hidden execution steps and governance, and latency is more than model speed due to routing overhead and granularity. The authors present an optimization-based approach that balances cost, quality, and latency, demonstrating its effectiveness on the AppWorld Test Challenge with a CodeAct agent.
  4. MosaicLeaks: Can your research agent keep a secret?This post introduces MosaicLeaks, a new benchmark and task for evaluating privacy leakage in deep-research agents that use external tools (like web search) in conjunction with private local documents. It identifies a critical failure mode where individually benign web queries can reveal sensitive information through a 'mosaic effect'. The post proposes a new training methodology, Privacy-Aware Deep Research (PA-DR), which uses situational rewards to train agents to balance task success with minimizing information leakage. PA-DR significantly reduces answer/full-information leakage from 34.0% to 9.9% while improving strict chain success from 48.7% to 58.7%. The work highlights the tension between informative queries for task performance and privacy, and demonstrates that simply prompting agents not to leak is insufficient.
  5. Agentic Resource Discovery: Let agents searchThis post introduces the Agentic Resource Discovery (ARD) specification, a draft open standard for how AI agents find and connect to capabilities across federated registries. Hugging Face's Discover Tool is presented as a reference implementation, integrating the Hub's semantic search to provide ARD catalog entries for Spaces, Agent Skills, and MCP Servers. The post details the `ai-catalog.json` manifest format and the `/search` registry API, and demonstrates usage via the `hf discover` CLI command and direct REST API calls. It emphasizes the importance of verification alongside discovery.
  7. Is it agentic enough? Benchmarking open models on your own toolingThis post introduces a benchmark harness for evaluating agentic use of command-line tools, with a specific focus on the `transformers` library. It details a methodology for measuring the effort (tokens, time, errors) agents expend to complete tasks, beyond just the final answer's correctness. The harness supports different tiers of tool access (bare, clone, skill) and introduces scoring metrics like 'match %', median time, median tokens, error rates, and 'marker adoption' to assess agent performance and guide tool optimization for agentic workflows. The implementation leverages Hugging Face Jobs for parallel execution and a Hugging Face Bucket for results storage.
  8. How an Agent Built a 3D Paris Gallery by Chaining Two Hugging Face SpacesThis post demonstrates an agent autonomously building a 3D Paris gallery by chaining two Hugging Face Spaces: TripoSplat for 3D model generation from a single image, and a Gradio Space for interactive gallery display. This showcases the practical application of the CLI for AI agents in orchestrating complex, multi-step workflows involving different Hugging Face capabilities.
  9. The Open Source Community is backing OpenEnv for Agentic RLThis post announces the transition of OpenEnv to a community-governed project under Hugging Face, with a focus on its role as an interoperability layer for RL environments. It standardizes environment publishing, deployment, and consumption via a Gymnasium-style API over HTTP/WebSocket, with Docker packaging and MCP compatibility. The post details the governance change, the protocol layer definition, and future development plans including external rewards, tasksets via datasets, harness integration, end-to-end examples, and auto-validation. The core contribution is the formalization and community adoption of OpenEnv as a standard for agentic RL environments.
  10. Designing the hf CLI as an agent-optimized way to work with the HubThis post details the re-architecture of the `hf` CLI to support both human and AI agent usage. Key changes include agent-specific output formats (TSV for agents, rich tables for humans), next-command hints for chaining operations, non-blocking commands that are safe to retry, and discoverable commands with examples. Benchmarking shows a significant reduction in token usage for agents when using the `hf` CLI compared to manual SDK/curl usage. The post also introduces the concept of the `hf-cli` skill for agents.
  11. ScarfBench: Benchmarking AI Agents for Enterprise Java Framework MigrationIntroduces ScarfBench, a benchmark for evaluating AI agents' capabilities in migrating enterprise Java frameworks. It defines key evaluation criteria including code understanding, transformation, and validation, and discusses the challenges and potential of AI agents in this domain.
  12. Inside VAKRA: Reasoning, Tool Use, and Failure Modes of AgentsThis post introduces VAKRA, a new benchmark for evaluating AI agents' reasoning and tool use in enterprise-like environments. It details the benchmark's structure, including task descriptions (API chaining, tool selection, multi-hop reasoning), evaluation framework, and analysis of observed failure modes. The benchmark comprises over 8,000+ locally hosted APIs across 62 domains, with tasks requiring 3-7 step reasoning chains. It highlights specific tools and API collections used (SLOT-BIRD, SEL-BIRD, REST-BIRD) and discusses challenges like API list length restrictions for agents.
  13. OpenEnv in Practice: Evaluating Tool-Using Agents in Real-World EnvironmentsThis post details the practical evaluation of tool-using AI agents in real-world environments using the Calendar Environment Server. It introduces the concept of a 'Calendar Environment Server' which acts as a simulated environment for agents to interact with external tools and APIs, specifically focusing on calendar management tasks. The post discusses the challenges and methodologies for evaluating agent performance in such interactive settings, including metrics for task completion, tool usage correctness, and overall agent effectiveness. It highlights the importance of robust evaluation frameworks for complex agentic systems.
  14. Introducing ⚔️ AI vs. AI ⚔️ a deep reinforcement learning multi-agents competition systemIntroduces AI vs. AI, a deep reinforcement learning multi-agents competition system, showcasing agentic workflows and competition systems. This extends the CLI for AI Agents by providing a concrete example of agent competition and interaction.
  15. Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP AgentsThis post introduces ProvenanceGuard, a source-aware verification layer for Model Context Protocol (MCP) agents. It addresses the failure mode of cross-source conflation by ensuring that claims made by an agent are attributed to the correct source, rather than just being supported by any source in the pooled evidence. ProvenanceGuard processes agent traces to decompose answers into claims, identify relevant sources, verify claim support using NLI models, and check attribution against the answer's stated or implied source. It also includes a repair loop for blocked answers. The system was evaluated on a medical agent and demonstrated high accuracy in catching unsupported claims and source attribution errors, with a focus on conservative decision-making for data-sensitive review.