New in llama.cpp: Model Management
Dec 11, 2025

Summary
Introduced router mode to llama.cpp server, enabling dynamic model management (loading, unloading, switching) without server restarts. Implemented a multi-process architecture for model isolation. Added auto-discovery of GGUF models from cache or a specified directory (`--models-dir`). Implemented on-demand loading and LRU eviction (`--models-max`). Added API endpoints for listing (`/models`), loading (`/models/load`), and unloading (`/models/unload`) models. Introduced support for per-model settings via presets.ini.