Blogs›Lambda›Dynamic GPU Resource Scaling for ML Competitions
The platform now supports dynamic scaling of GPU resources, allowing additional GPUs to be added mid-competition without interrupting ongoing battles. This is achieved by independently serving models on each GPU with vLLM, using Caddy for load balancing and health checks, and a dynamic concurrency limiter that resizes based on available resources. This enables efficient cost management and ensures competition integrity even with fluctuating demand. This post highlights the increasing demand for serving large open-weight models like GLM 5.2, emphasizing the infrastructure challenges and the need for efficient serving solutions to keep up with the capability gap closing between open and closed-weight models.
7/30/2026
Introduced a system for dynamically adding GPUs to an ML competition platform mid-round. This involved configuring independent vLLM replicas per GPU, using Caddy as a central proxy and forward proxy with an egress allowlist, and implementing a runtime-resizing concurrency limiter. The process ensures new GPUs are integrated seamlessly by first updating Caddy's routing and then adjusting the worker concurrency budget.
7/9/2026
This post discusses the increasing adoption and performance of the open-weight GLM 5.2 model, noting its use in intensive research pipelines and as a subagent in complex AI workflows. It highlights the infrastructure requirements for serving such large models at full quality, including significant VRAM and fast GPUs, and positions efficient model serving as the new bottleneck as open-weight models close the capability gap with proprietary ones. It also references a linked post on deploying GLM 5.2 on Lambda, implying infrastructure-level support for these models.