Secure LLM Inference Access Control

secure-llm-inference-access-control: Introduces adapter-based access control for LLM inference, enabling fine-grained, per-query authorization by attaching lightweight, domain-specific model extensions (adapters) to a single shared base model. This approach addresses the challenge of preventing LLMs from leaking confidential information by enforcing access policies at inference time, rather than relying on post-hoc filtering or costly private silos. The secure-inference project integrates with the llm-d framework, utilizing LoRA adapters for each domain with policy-enforced routing. This post details the training and inference phases, including user authentication via JWT/SSO, authorization via Open Policy Agent, semantic matching for adapter selection, and routing to inference pools with appropriate adapters loaded. It provides an example of per-role inference routing for a software engineer.