How to stop your autoresearch loop from cheating

What this post added

This post details experiments using Andrej Karpathy's autoresearch framework to autonomously optimize AI models. It describes the implementation of a bash harness for Codex to enable looping and A/B testing in training optimization, and the application of autoresearch to dynamic expert swapping for inference optimization on consumer GPUs. The post analyzes the results, emphasizing the convergence of different agents on similar optimizations, the impact of proposal quality on cost, and the critical role of environment design and task framing in productive exploration. It also identifies infrastructure gaps in sandboxing and tooling that hinder autonomous research.