TechPrufer
Get started
Google
Google logoGoogle · Feature trail

Serialization Libraries and Performance

4 posts · Jul 2008 – Aug 2024

What this trail covers

Google's development and promotion of serialization libraries focused on memory efficiency and performance. It began with the introduction of FlatBuffers, a library designed for zero-copy deserialization, enabling faster data access and reduced memory overhead. Subsequent developments have included the release of Protocol Buffers as open source, providing a language-neutral, platform-neutral, extensible mechanism for serializing structured data, and the introduction of Zopfli for denser data compression.

All Google trails
  1. Streamlining LLM Inference at the Edge with TFLite- Google Developers BlogIntroduced an improved XNNPack weight cache provider for TensorFlow Lite that utilizes memory mapping (mmap) to load packed weights from disk. This eliminates the need for repacking weights on each process start, significantly reducing startup latency and peak memory usage. It also enables weight sharing across processes and simplifies the user-facing API by allowing a cache file path to be provided. The post details the new XNNPack cache provider interface and demonstrates performance improvements through benchmarks.
  2. Announcing Kotlin support for protocol buffers- Google Developers BlogIntroduced native Kotlin support for Protocol Buffers, enabling developers to use Protocol Buffers directly within Kotlin projects without requiring Java interop. This simplifies the development workflow for Kotlin developers working with Protocol Buffers, improving efficiency and reducing boilerplate code.
  3. FlatBuffers 1.1: a memory-efficient serialization library- Google Developers BlogThis post introduces FlatBuffers 1.1, highlighting its memory-efficient serialization capabilities and its advantage of zero-copy deserialization. It emphasizes the library's suitability for performance-critical applications and its role in improving data access speed and reducing memory footprint.
  4. Compress data more densely with Zopfli- Google Developers BlogIntroduced Zopfli, an optimization of the Deflate compression algorithm that achieves higher compression ratios (up to 8%) than standard Deflate, at the cost of significantly increased compression time. This is beneficial for scenarios where compression time is less critical than the resulting file size, such as for static assets or archival.
  5. Protocol Buffers, our serialized structured data, released as Open Source- Google Developers BlogThis post announces the open-sourcing of Protocol Buffers, a language-neutral, platform-neutral, extensible mechanism for serializing structured data. It highlights its efficiency and suitability for inter-process communication and data storage.