Open-sourcing Baseten’s suffix automaton MTP accelerator

Introduced a hybrid speculative decoding approach combining suffix automata (SA) with multi-token prediction (MTP) for LLM inference. This approach leverages SA for long pattern matching and MTP for general speedups, with a threshold to switch between them. The implementation is integrated into NVIDIA TensorRT-LLM, with SA states updated directly on the device without additional synchronization points. This results in up to 40% higher throughput or 40% lower latency on agentic coding workloads, with near-zero overhead and full GPU utilization.