Lua Runtime in Go

Shopify has evolved its event processing capabilities by implementing the Go worker pool pattern to control concurrency and improve performance. This has led to a significant increase in event processing throughput, enabling better preparation for high-traffic events and reuse of the pattern across different pipeline components. This post details the development of go-lua, a pure Go implementation of the Lua programming language, to serve as the core execution engine for their load generation tool. The implementation manually ported the Lua 5.2 compiler and virtual machine from C to Go, offering advantages in garbage collection, minimal execution context overhead, and easier concurrency mapping. Challenges included optimizing the interpreter's inner loop performance due to Go's switch statement implementation and limited function call inlining, leading to the exploration of function table dispatch and manual inlining. Benchmarks showed go-lua to be slower than C Lua but competitive with other Go implementations, with limitations including lack of coroutines, string pattern matching, and weak reference tables.