Architecting offline-first generative AI applications for edge deployments using AWS services | Amazon Web Services

What this post added

This post introduces a reference architecture for offline-first generative AI applications at the edge. It details the end-to-end pattern from cloud model customization (Amazon Bedrock, SageMaker AI) to edge deployment (AWS IoT Greengrass) and local inference (Strands Agents). Key contributions include: elaborating on model customization strategies (fine-tuning, continued pre-training, RAG, and hybrid approaches), discussing architectural decisions based on use case requirements and hardware constraints, and presenting a reference architecture diagram. It also covers specific deployment strategies for the gpt-oss-20b model on edge hardware, including model replication and tensor parallelism, and highlights the integration of AWS services like Amazon S3, Amazon Nova Pro, and AWS IAM.