Hugging Face

New in llama.cpp: Model Management

Dec 11, 2025
New in llama.cpp: Model Management
SummaryAI, from the source post

Introduced router mode to llama.cpp server, enabling dynamic model management (loading, unloading, switching) without server restarts. Implemented a multi-process architecture for model isolation. Added auto-discovery of GGUF models from cache or a specified directory (`--models-dir`). Implemented on-demand loading and LRU eviction (`--models-max`). Added API endpoints for listing (`/models`), loading (`/models/load`), and unloading (`/models/unload`) models. Introduced support for per-model settings via presets.ini.

More from the Local AI Inference Stack trail