Cloudflare

Rust streaming HTML parser/rewriter

Nov 29, 2019Andrew Galloni, Ivan Nikulin
Rust streaming HTML parser/rewriter
SummaryAI, from the source post

This post details the development of LOL HTML, a Rust-based streaming HTML rewriter/parser with a CSS-selector based API, which serves as the backend for Cloudflare Workers' HTMLRewriter. It introduces a dual-parser architecture (lexer and tag scanner) to optimize performance by skipping non-matching content and avoiding expensive token propagation to the Workers runtime. It also describes a Rust macro-based DSL for defining Nondeterministic finite automata for parsing and byte slice processing optimizations using token outlines for lazy transformation into byte slices.

More from the Edge Computing with Cloudflare Workers trail