TechPrufer
Get started
Hugging Face
Hugging Face logoHugging Face · Feature trail

Kubernetes Deployment of Vision Transformers with TF Serving

3 posts · Jul 2022 – Aug 2022

What this trail covers

Hugging Face now supports deploying Vision Transformer (ViT) models on Google Cloud's Vertex AI platform, offering a scalable and managed solution for ML model serving. This builds upon previous posts detailing local deployment and Kubernetes deployment, providing a streamlined workflow with less code. Vertex AI offers features like autoscaling, model versioning, traffic splitting, rate limiting, and monitoring. The integration leverages pre-built TensorFlow serving containers and the `google-cl`, and also supports local deployment of TensorFlow Vision Models using TensorFlow Serving, exposing them as REST or gRPC endpoints with features like model warmup and server-side batching.

All Hugging Face trails
  1. Deploying 🤗 ViT on Vertex AIThis post details the deployment of Hugging Face's Vision Transformer (ViT) models on Google Cloud's Vertex AI platform. It outlines the workflow, including uploading the SavedModel to Vertex AI's model registry, creating a Vertex AI Endpoint, and deploying the model to the endpoint. It highlights the use of pre-built TensorFlow serving containers and the `google-cloud-aiplatform` Python SDK for managing the deployment lifecycle. The post also covers performing predictions, monitoring with Vertex AI, and local load testing.
  2. Deploying 🤗 ViT on Kubernetes with TF ServingThis post details the deployment of Vision Transformer (ViT) models on Kubernetes using TensorFlow Serving. It provides a concrete example of how to containerize a ViT model and deploy it as a service within a Kubernetes cluster, leveraging TF Serving for efficient inference. The post likely includes steps for creating Docker images, Kubernetes deployment manifests, and service configurations to enable scalable and accessible deployment of ViT models.
  3. Deploying TensorFlow Vision Models in Hugging Face with TF ServingThis post details how to deploy a Vision Transformer (ViT) model locally using TensorFlow Serving (TF Serving). It covers saving the model in the SavedModel format, performing model surgery to embed preprocessing (image scaling, normalization, resizing) and postprocessing (mapping logits to ImageNet-1k classes) into the computation graph, and exporting the model with a signature that accepts string inputs for images. It then demonstrates deploying the model with `tensorflow_model_server` and querying the REST and gRPC endpoints.