Compose Readers and Writers in Golang applications
Feb 15, 2024Igor Drozdov

Summary
This post details how GitLab Workhorse utilizes Golang's `io.Reader` and `io.Writer` interfaces and their composite functions (`io.Copy`, `io.MultiWriter`, `io.MultiReader`, `io.TeeReader`) to efficiently handle streaming of large Git data. Specific examples include streaming `InfoRefs` RPC responses via HTTP and implementing the Dependency Proxy by caching upstream images to object storage while simultaneously serving them to users. The post explains the underlying Go interfaces and functions and provides code examples from the GitLab codebase.