Making Kimi K3 tokenization 18x faster for million-token agentic workloads

Introduced the Baseten Tokenizer (Basetenkenizer), a Rust-based implementation for Kimi K3 tokenization, achieving up to 18x speedup for million-token sequences compared to Python's tiktoken. The Basetenkenizer consolidates special-token scanning, BPE merging, and safe chunking into a single native call, optimizing for long input sequences in agentic workloads. Key optimizations include specialized pre-tokenization scanners, a stack-resident BPE merge tier, multi-core parallelism for identical pre-tokens, native typed segments with safe chunking in Rust, zero-copy NumPy ownership transfer, and smart-pointer PyO3 bindings. Benchmarks demonstrate significant latency reduction for the serving path, especially with high prefix cache hit rates.