Empowering Engineers with AI

This post details the implementation of an AI bot for resolving escalations in Slack channels, using LLMs for post classification into six categories (ANNOUNCEMENT, HOW_TO_REQUEST, HUMAN_ACTION, PULL_REQUEST_REVIEW, SERVICE_ISSUE_REQUEST, UNKNOWN) and Amazon Bedrock Knowledge Bases to import technical documentation, historical channel posts, and Canvas content. It measures 30% positive user ratings and estimates 3,000+ hours saved. It also describes custom AI assistants in Slack (14% internal LLM traffic), a web app for advanced agent interaction (37% internal LLM traffic), and IDE integrations for AI assistance. A unified LLM platform is detailed with features like multi-LLM access, a defined schema, a derived data system for metrics, and knowledge base APIs. Tools like CLI tools, an 'Evaluate' playground, and a 'Compare' tool are also described to assist in AI development efforts. The post estimates 10,000+ engineering hours saved by year-end, accounting for information discovery, documentation reading, context switching, comprehension, translation, and hallucination time.