tokenizers v1: encode, decode and scaling, measured

This post details the performance improvements in version 1 of the tokenizers library. It introduces several key optimizations: a workspace split into `tk-encode` (runtime) and other modules (serialize, convert, train) for modularity; a no-alloc merge working set that uses a caller-owned scratch buffer; the 'bitcannon' approach which uses boolean operations over bitstreams and SIMD instructions for faster splitting instead of regex; a rewrite of the merge loop using an intrusive doubly-linked list within a preallocated buffer and batch processing of pre-tokens; a thread-local word cache to memoize results for repeated pre-tokens; and native parallelism allowing multiple threads to encode concurrently by drawing scratch buffers and word caches from their own sub-pools. The post also outlines the benchmarking methodology used to measure these improvements.