Replicate Feature Trails

Feature trailActive

Video Generation Prompting and Control

Replicate continues to expand its video generation capabilities with support for advanced models like Veo 3 and Wan 2.1. This post details how to craft effective prompts for Veo 3, focusing on specifying visual elements (subject, context, action, style, camera motion, composition, ambiance) and audio elements (dialogue, ambient noise, sound effects, music). It highlights Veo 3's unique consistency, where similar prompts yield similar results, which can be leveraged for character consistency. This post introduces FLUX 3, a multimodal foundation model trained on images, audio, and video, aiming to encapsulate a more comprehensive understanding of reality. It showcases FLUX 3's capabilities in text-to-video generation with detailed prompts, image-to-video transformation using start and end frames, video continuation from a base clip, generating multiple scenes and camera angles within a single prompt, and achieving diverse artistic styles beyond hyper-realism. Examples include VHS-style home videos and zero-gravity simulations.

5 posts · 2024–2026

Timeline

20242026

Feature trailActive

Language Model Deployment and Fine-tuning

Replicate continues to expand its support for large language models, now including fine-tuning capabilities for Llama 2 and new models like StableLM, Vicuna, and GPT4All. The platform provides tools for creating custom models, authenticating API access, and initiating training jobs, with the ability to monitor progress and run the fine-tuned models via API or web interface. Playgrounds like OpenPlayground and AI Playground are emerging to compare and tune these models. This post introduces the Llama 3.1 405b model and details how to fine-tune FLUX.1 for personalized image generation, enabling users to create images of themselves in various scenarios using a simple web interface or API. The platform also highlights the ease of use for fine-tuning faces compared to previous models and provides guidance on prompt engineering with LLMs.

21 posts · 2022–2026

Timeline

20222026

Feature trailActive

Image Generation Optimization

Replicate continues to expand its offerings in image generation with support for new models like Ideogram v2 (including a Turbo variant for faster generation) and Stable Diffusion 3.5. This post details the capabilities of Ideogram v2, highlighting its inpainting feature and strong performance in generating text within images. It provides API examples for using the model and links to an open-source inpainting demo. Tips for achieving the best inpainting results are also included, emphasizing pro. This post details how to get the best results from Stable Diffusion 3 Medium, focusing on its advanced text encoders, prompting techniques (including longer prompts and avoiding negative prompts), and optimal settings for resolution, steps, and guidance scale. It also discusses different SD3 model versions based on VRAM availability and the impact of various parameters on image quality and coherence.

26 posts · 2022–2025

Timeline

20222026

Feature trailActive

Krea 2 Model Aesthetics

Replicate continues to expand its offerings in image generation with the introduction of Nano Banana Pro, a model focused on advanced logic, text adherence, and character consistency. The model demonstrates an ability to interpret and respond to textual information within images, generate correct answers with work shown for homework problems, and create summary images from long pieces of information. It also excels at rendering code and maintaining pixel-perfect text accuracy across various styles. The model can process up to 14 reference images simultaneously, allowing for consistent character appearances, poses, and styles across different scenes and contexts.

1 post · 2025

Timeline

20252026

Feature trailActive

Document Parsing and OCR

Replicate now offers advanced document parsing and text extraction capabilities through the integration of Datalab Marker and OCR models. Marker processes various document formats (PDF, DOCX, PPTX, images) into markdown or JSON, handling tables, math, and code, and supporting structured field extraction via JSON Schema. OCR detects text in 90 languages from images and documents, providing reading order and table grids. Both models offer high performance, outperforming established tools like Tesseract, with short processing times. Marker processes a page in about 0.18 seconds and can hit 120 pages per second when batched. Structured extraction is a powerful feature of Marker, allowing specific fields to be extracted from documents like invoices. Performance evaluations show Marker outperforming other models on benchmarks like olmOCR-Bench.

1 post · 2025

Timeline

20252026

Feature trailCooling

LLM API Access

Replicate provides API access to large language models, including Snowflake's Arctic, Meta's Llama 3, IBM's Granite 4.0, and Meta's Llama 3.1 405B. The platform offers client libraries for JavaScript, Python, and other languages, as well as direct HTTP API access. Users can stream output and integrate these models into their applications. The platform also emphasizes responsible AI practices and provides example applications and API playgrounds for various models. Arctic is a 480B parameter model. This post details running Llama 3 with API access, including client libraries for JavaScript and Python, and direct HTTP API calls via cURL. It also outlines the different Llama 3 variants available and provides a link to a demo chat app.

1 post · 2024–2025

Timeline

20242026

Feature trailCooling

Torch Compile Caching

Replicate has implemented caching for `torch.compile` artifacts to significantly reduce model boot times for PyTorch-based models. This optimization caches compiled code across model container lifecycles, leading to 2-3x faster cold starts for models like those in the FLUX family. The system keys cache files on model version and stores them near GPU nodes, updating the cache upon graceful container shutdown. This has been further extended to dramatically improve cold boot times for fine-tuned models.

2 posts · 2023–2025

Timeline

20232026

Feature trailCooling

Search API

Replicate has introduced a new Search API to help users find models, collections, and documentation. This API provides enhanced search capabilities with richer metadata, including generated descriptions and tags. It is integrated into the TypeScript and Python SDKs, and the MCP servers, with sophisticated response filtering to optimize LLM context windows. The older search endpoint remains active but is deprecated in favor of the new API. This post introduces the Model Context Protocol (MCP) server, a hosted remote service that allows language models to interact with Replicate's HTTP APIs, enabling natural language querying, comparison, and execution of models. It also details JSON response filtering using `jq` via WebAssembly for managing large API responses and highlights secure authentication with Cloudflare Workers.

2 posts · 2025

Timeline

20252026

Feature trailCooling

Video Style Transfer with LoRAs

Replicate continues to expand its support for fine-tuning open-source video models, specifically adapting LoRA-based style transfer for models like HunyuanVideo. This allows users to train custom video generation models on their own visual content, enabling personalized video stylization with in-motion style transfer. The process involves using dedicated training models and then generating videos via the browser, Playground, or API, with optimizations for faster inference. This post highlights new capabilities for fine-tuning HunyuanVideo using the Musubi Tuner, enabling in-motion style transfer and character animation.

7 posts · 2023–2025

Timeline

20232026

Feature trailCooling

GPU Hardware Support and Migration

Replicate has introduced support for NVIDIA L40S GPUs, offering significant performance improvements over A40 GPUs. This includes a migration plan for existing models and deployments from A40s to L40S, with updated pricing considerations. Users are advised to adjust minimum instance counts for deployments to account for the increased performance per dollar. Support for A40 GPUs will be phased out.

3 posts · 2024–2025

Timeline

20242026

Feature trailQuiet since 2024

AI Model Training Data Curation

Replicate continues to expand its offerings in AI model development by highlighting advancements in data curation and synthetic data generation. This includes techniques for boosting multimodal learning efficiency, creating massive filtered educational datasets, and automating the generation of function-calling datasets. The platform also showcases tools for training custom classifiers with single prompts and emphasizes the importance of data flywheels for self-improving LLM applications. This post introduces FineWeb-Edu, a 1.3 trillion token dataset of high-quality educational web content, and APIGen, an automated system for generating high-quality function-calling datasets.

1 post · 2024

Timeline

20242026

Feature trailQuiet since 2024

Local ML Model Deployment

Replicate enables users to run advanced ML models locally on their own hardware, starting with Stable Diffusion on Apple Silicon Macs. This includes providing detailed instructions for cloning repositories, setting up environments, and running models, as well as exploring alternative CLI tools like DiffusionKit for easier deployment. The focus is on empowering users with local inference capabilities for powerful generative AI models. This post extends this by providing instructions for running Stable Diffusion 3 locally using ComfyUI and Cog.

1 post · 2022–2024

Timeline

20222026

Feature trailQuiet since 2024

Machine Learning Tooling and Deployment

Replicate democratizes machine learning by providing tools and infrastructure that abstract away complexities like GPUs, CUDA, and dependency management. Cog, 'Docker for machine learning,' allows models to be packaged into standard containers for easy sharing and deployment, enabling software engineers to integrate ML capabilities without deep ML expertise. This post details how to package and deploy custom versions of models like Stable Diffusion 3 using Cog. The platform also offers API access.

1 post · 2022–2024

Timeline

20222026

Feature trailQuiet since 2024

Network Security and Isolation

Replicate has implemented enhanced network security measures for model containers, including encrypting all internal traffic and restricting privileged network access. This follows the disclosure of a vulnerability where model containers could eavesdrop on internal communications due to shared network namespaces and unencrypted traffic. The company has mitigated this by dropping NET_ADMIN and NET_RAW capabilities from model containers and encrypting all internal TLS traffic.

1 post · 2024

Timeline

20242026

Feature trailQuiet since 2023

Language Model Streaming Output

Replicate now supports server-sent event streams for language models, allowing for live, word-by-word updates in applications. This feature is integrated into the API and client libraries, enabling developers to build more interactive experiences like chat applications. The post details how to enable streaming for predictions, provides Node.js and cURL examples, lists supported models, and explains how to add streaming support to custom Cog models.

1 post · 2023

Timeline

20232026

Feature trailQuiet since 2023

Incident Response and Observability

Replicate has launched a public status page and is publishing detailed incident reports to improve transparency and communicate system health. The first report details a significant outage on May 11th caused by connection pool exhaustion due to stacked INSERT queries on their PostgreSQL database, triggered by new asynchronous prediction update features. The incident highlights the challenges of complex systems failures, the interplay between performance improvements and new hazards, and the need for robust observability and incident response.

1 post · 2023

Timeline

20232026

Feature trailQuiet since 2022

Image Collection and Retrieval

Replicate has introduced tools and techniques for automating image collection and retrieval, leveraging CLIP embeddings and k-NN search over large datasets like LAION-5B. This enables users to curate custom datasets for fine-tuning models, steer text-to-image generation with specific aesthetics, and find images similar to a given input image. The `clip-retrieval` Python package is highlighted as a key tool for this capability, with integrations into Replicate's platform for running text-to-image generation and for finding images similar to a given input image.

1 post · 2022

Timeline

20222026

Feature trailQuiet since 2022

CLIPDraw Constrained Generation

Replicate continues to expand its capabilities in generative AI by enabling more fine-grained control over model outputs. This post details how to constrain the CLIPDraw model to encourage drawing in the center of the canvas, demonstrating techniques for modifying loss functions to influence spatial distribution and detail. This builds upon existing capabilities in image generation and model customization.

1 post · 2022

Timeline

20222026