TechPrufer
Sign inGet started

Blogs›Google›Memory-Efficient Serialization Libraries

Memory-Efficient Serialization Libraries

Memory-Efficient Serialization Libraries

2
posts
2015–2024

Google's development and promotion of memory-efficient serialization libraries, focusing on enabling high-performance data handling for applications. Early efforts introduced FlatBuffers, a serialization format designed for performance-critical applications, allowing direct access to data without parsing or unpacking. Subsequent developments have seen the release of FlatBuffers 1.1, which further enhances its capabilities and usability for developers. This has now been extended to include an improved XNNPack weight cache provider for TensorFlow Lite, leveraging memory-mapped files (mmap) for persistent storage and cross-process sharing of packed weights. This significantly reduces startup latency, peak memory usage, and eliminates the need for repeated weight repacking, especially beneficial for LLMs and edge AI inference.

2024

Streamlining LLM Inference at the Edge with TFLite- Google Developers Blog

8/13/2024

Introduced an improved XNNPack weight cache provider for TensorFlow Lite. This provider utilizes memory-mapped files (mmap) to persist packed weights to disk, enabling them to be loaded across multiple TFLite interpreter instances and even across processes. This eliminates the overhead of repacking weights on each model load, significantly reducing startup latency and peak memory usage. It also allows for weight deduplication for models with shared subgraphs and simplifies the user-facing API by requiring only a cache file path.

2015

FlatBuffers 1.1: a memory-efficient serialization library- Google Developers Blog

4/2/2015

This post announces FlatBuffers 1.1, a new version of the memory-efficient serialization library. Key improvements include the addition of a new C++ API for building schemas, support for C++11, and enhancements to the Java API. It also highlights the library's use in Google's internal systems and its benefits for performance-critical applications.