Google Cloud Datastore

4/29/2026

Introduced Rapid Bucket, a high-performance object storage solution leveraging Google's Colossus architecture and gRPC bidirectional streams for AI/ML workloads. Integrated Rapid Storage with PyTorch via `fsspec` and `gcsfs`, enabling significant performance gains (23% reduction in training time, 4.8x read throughput, 2.8x write throughput) by bypassing legacy REST APIs and utilizing zonal co-location for reduced latency. This integration requires no code rewrites for existing `fsspec` clients.

4/8/2025

Introduced Managed I/O for Google Cloud Dataflow, which automates the management of Apache Beam I/O connector versions, offers standardized Java and Python APIs for sources and sinks (e.g., Iceberg, Kafka, BigQuery), and automatically optimizes connectors for the Dataflow runner. Benchmarks demonstrate improved performance and efficiency for batch and streaming pipelines.

11/13/2024

This post introduces the integration of vLLM, an open-source library for high-throughput LLM inference, with Google Cloud Dataflow's model manager. It details how vLLM's continuous batching technique significantly improves LLM inference performance compared to traditional batching, achieving over a 23x improvement in vCPU hours for processing prompts. The post explains how Dataflow's model manager simplifies the deployment of vLLM within streaming pipelines by allowing users to control the exact number of model copies, thereby optimizing resource utilization and reducing out-of-memory issues for large models.

8/16/2024

This post introduces the integration of Gemma, a lightweight LLM, with Google Cloud Dataflow for real-time streaming ML tasks. It details a pipeline that uses Gemma for sentiment analysis and summarization of customer chat data, with negative sentiment chats triggering AI-generated response drafts for human review. The post highlights the benefits of embedding LLMs directly into streaming pipelines for data locality, unified autoscaling, and simplified observability, and provides code examples for setting up Gemma with Dataflow ML, including custom model handlers and prompt engineering.

8/15/2022

This post details the technical process and considerations for migrating from App Engine Blobstore to Cloud Storage. It covers the rationale for migration, the challenges encountered, and the steps taken to ensure a smooth transition, including data migration strategies and application code updates. The post emphasizes the benefits of Cloud Storage for scalability, performance, and cost-effectiveness compared to the legacy Blobstore.

6/14/2022

This post details the migration of App Engine Memcache to Cloud Memorystore, a managed Redis service. It covers the technical considerations, challenges, and steps involved in this migration, including strategies for minimizing downtime and ensuring data integrity. The migration process involves understanding the differences between the two services, planning the transition, and executing the switchover. Cloud Memorystore offers enhanced performance, scalability, and features compared to App Engine Memcache, making it a suitable successor for caching needs.

8/4/2021

This post details the process and considerations for migrating from Cloud NDB to Cloud Datastore, focusing on the technical challenges and solutions involved in moving data and application logic between these two Google Cloud services. It covers aspects of data modeling, query translation, and performance tuning during the migration process.

7/21/2021

This post details the migration of App Engine's ndb client library to Cloud NDB. It covers the process of migrating from the legacy App Engine Datastore client to the new Cloud NDB client, including considerations for data consistency, performance, and the use of features like eventual consistency and strong consistency. The migration involves updating dependencies, adapting code to the new client's API, and testing the application's behavior under different consistency models.

2/1/2019

Announces the general availability of Cloud Firestore, a new NoSQL document database for serverless applications. Highlights its real-time data synchronization, offline support, and querying capabilities. Mentions that it builds on the foundation of the Datastore API but offers a distinct set of features and improvements.

10/3/2017

This post introduces Cloud Firestore, a new document database for apps, designed to be a scalable, serverless, NoSQL cloud database. It offers real-time data synchronization, offline support, and automatic scaling, making it suitable for mobile, web, and IoT applications. Key features include a flexible document data model, powerful querying capabilities, and seamless integration with other Firebase services.

2/20/2014

This post introduces how to create a blog on App Engine using Django and Google Cloud SQL, demonstrating the integration of these services for building web applications. It details the setup process, including configuring the Django project, setting up the Cloud SQL instance, and deploying the application to App Engine.

2/11/2014

This post announces the General Availability of Google Cloud SQL, a managed relational database service. Key advancements include an Service Level Agreement (SLA) guaranteeing uptime, support for larger database sizes up to 500GB, and the introduction of encryption at rest for enhanced data security. These features aim to provide a more robust, scalable, and secure managed database solution for developers.

8/7/2013

This post announces enhancements to Google Cloud Platform services, specifically Compute Engine and App Engine, and updates to Cloud Datastore. For Compute Engine, it highlights improvements in instance management and networking. For App Engine, it details new features for custom runtimes and scaling. For Cloud Datastore, it mentions performance optimizations and new query capabilities, further solidifying its role as a scalable NoSQL database.

5/16/2013

This post introduces Google Cloud Datastore as a fast, powerful, NoSQL database. It highlights its schema-less nature, automatic scaling, and high availability, positioning it as a solution for building robust applications. The post aims to guide developers on getting started with Datastore.

5/9/2013

This post introduces a new API that allows developers to save files directly to Google Drive, simplifying the process of integrating Google Drive functionality into applications. This enhancement aims to improve developer workflows and user experience by providing a more direct and accessible way to manage files within the Google Drive ecosystem.

4/4/2013

This post announces the expanded availability of Google Compute Engine, including new features and lower prices. It highlights the ability to run custom machine types, the introduction of preemptible instances for cost savings, and improved networking capabilities. The post also mentions the integration of Compute Engine with other Google Cloud services, emphasizing its role as a foundational infrastructure component.

3/18/2013

This post introduces Tailbone, a JavaScript library that enables direct communication between client-side JavaScript and Google App Engine, specifically targeting the Datastore. It details how to set up and use Tailbone for CRUD operations on the Datastore, including handling authentication and data serialization, thereby extending App Engine's accessibility to front-end developers.

11/29/2012

This post announces enhancements to Google Cloud Storage, focusing on improved performance and value for developers. It highlights features like multi-region buckets for higher availability and durability, and introduces new pricing tiers that offer more value for performance-sensitive workloads. The post also mentions the deprecation of the older Google Cloud Storage XML API in favor of the newer JSON API, encouraging migration for better performance and feature access.

11/8/2012

This post introduces the ability to get started with Cloud SQL at no cost, offering a faster and larger database option. It highlights the benefits of Cloud SQL for developers, including its ease of use, scalability, and performance, positioning it as a compelling choice for new projects.

10/9/2012

This post details how Streak integrated CRM functionality into the inbox by leveraging Google Cloud Platform. It highlights the use of Google Cloud's infrastructure for data storage and management, enabling a seamless connection between email and CRM capabilities. This demonstrates the platform's ability to support complex application integrations.

5/10/2012

This post announces the pricing plan for Google Cloud SQL, a managed relational database service. It details the pricing structure, including per-instance pricing and storage costs, and highlights the benefits of using Cloud SQL for developers, such as ease of use, scalability, and reliability. This contributes to the evolution of Google Cloud's data storage offerings by providing a clear pricing model for a key database service.

4/3/2012

This post introduces Google Cloud Storage, highlighting its high performance and ease of use for developers. It emphasizes the system's ability to handle large amounts of data efficiently and reliably, positioning it as a robust solution for various application needs.

3/20/2012

Introduced object versioning, object lifecycle management, and fine-grained access control to Google Cloud Storage, addressing highly requested features for enhanced data management and security.

1/17/2012

This post introduces concurrency controls for Google Cloud Storage, specifically focusing on conditional requests using If-Generation-Match and If-Generation-Not-Match headers. It also details deeper integration with App Engine, allowing developers to use Cloud Storage directly from their App Engine applications without needing to manage credentials manually, simplifying the development of data-intensive applications.

11/28/2011

This post introduces a new approach to access control in Google Cloud Storage, simplifying the management of permissions for buckets and objects. It details the introduction of signed URLs, which allow for temporary, time-limited access to objects without requiring users to have Google accounts or explicit permissions. This simplifies sharing and distribution of content stored in Cloud Storage.

10/11/2011

This post announces the release of Google Cloud Storage from Code Labs, indicating its transition to a more stable and generally available offering. It also highlights new features and a lower price point, suggesting ongoing development and cost optimization for the service.

10/6/2011

This post introduces Google Cloud SQL, a fully managed relational database service for applications. It highlights features like automated management, scaling, and backups for MySQL, PostgreSQL, and SQL Server, aiming to simplify database operations in the cloud.

5/10/2011

This post announces the general availability of Google Storage for Developers, previously in limited beta. It highlights new features including increased storage limits (up to 100GB per bucket), improved performance, and enhanced developer tools for managing data. The service is now open to all developers.

6/16/2010

This post introduces the ability to share Google Storage buckets with groups, enhancing access control and collaboration for developers. It details how to manage group permissions for storage access, building upon the existing features of Google Cloud Storage.

5/19/2010

This post announces a preview of 'Google Storage for Developers,' a new storage service designed to offer developers a robust and scalable solution. It highlights the service's potential for developers to store and manage their data effectively.

4/7/2009

This post introduces standards-based persistence for Java applications on Google App Engine, enabling developers to use Java Data Objects (JDO) and Java Persistence API (JPA) for data management. It highlights the benefits of these standards for building robust applications and outlines the initial steps for developers to adopt them.

12/16/2008

Introduced the App Engine System Status Dashboard, providing developers with real-time visibility into the operational status of the App Engine platform. This dashboard offers metrics and information to help developers monitor performance, identify potential issues, and understand the overall health of the service.

4/18/2008

This post introduces the ability to run App Engine applications locally, enabling developers to develop and test their applications offline. It highlights the benefits of local development for faster iteration and debugging, and provides instructions on how to set up and use the local development environment.