Own your AI: Self-Hosted GitLab Duo models with AWS Bedrock

This post details the implementation of GitLab Duo Self-Hosted models, specifically integrating with AWS Bedrock. It outlines the architecture involving a self-managed GitLab instance, an AI Gateway, and AWS Bedrock as the LLM backend. Key implementation steps include installing the AI Gateway (Docker or Helm), configuring GitLab to point to the AI Gateway, requesting access to AWS Bedrock models, configuring self-hosted models within GitLab Duo (specifying deployment name, platform, model family, and identifier), and assigning these models to specific GitLab Duo features (Code Suggestions, GitLab Duo Chat, etc.). Verification steps include running a health check, testing Duo features, and reviewing AI Gateway logs. The post also provides guidance on selecting the right model for each use case and mentions alternative deployment options like on-premises vLLM and Azure OpenAI Service.