TechPrufer
Get started
Mistral AI
Mistral AI logoMistral AI · Feature trail

Vision Language Model Fine-tuning

1 post · Jun 2024 – Aug 2025

What this trail covers

Mistral AI continues to enhance its vision and multimodal model capabilities. Pixtral Large, a 124B multimodal model built on Mistral Large 2, offered frontier-level image understanding, excelling in document, chart, and natural image analysis. It achieved state-of-the-art performance on benchmarks like MathVista, DocVQA, and MM-MT-Bench. The model supported a 128K context window, allowing for the processing of multiple high-resolution images. Mistral Large 2 also received an update with improved performance.

All Mistral AI trails
  1. Unlocking the potential of vision language models on satellite imagery through fine-tuning | Mistral AIThis post details the practical application of LoRA fine-tuning for vision language models (VLMs) using Pixtral-12B on satellite imagery. It quantifies performance improvements (accuracy increase from 0.56 to 0.91, hallucination reduction from 5% to 0.1%) achieved with a low budget and limited samples. It also outlines the fine-tuning process via API and UI, including hyperparameter selection guidance (learning rate, batch size, epochs) and highlights the limitations of prompt engineering for ambiguous classes. A cookbook link is provided for implementation details.
  2. [Deprecated] Pixtral Large | Mistral AIThis post announces Pixtral Large, a 124B multimodal model that extends Mistral Large 2. It details its performance on benchmarks like MathVista, DocVQA, and MM-MT-Bench, highlighting its state-of-the-art capabilities in image understanding, including documents and charts. The model features a 128K context window and a 1B parameter vision encoder. It also mentions an update to Mistral Large 2 with improved long context understanding and function calling. The post notes that Pixtral Large is deprecated and replaced by newer models.
  3. [Deprecated] Pixtral 12B | Mistral AIThis post announces Pixtral 12B, a multimodal model with a new 400M parameter vision encoder and a 12B parameter multimodal decoder based on Mistral Nemo. It supports variable image sizes and aspect ratios, and can process multiple images within a 128K token context window. Pixtral 12B achieves strong performance on multimodal tasks, including chart and document understanding, and maintains state-of-the-art performance on text-only benchmarks. The model is released under Apache 2.0. It is noted as deprecated and replaced by newer models.
  4. My Tailor is Mistral | Mistral AIThis post introduces model customization on la Plateforme, enabling users to adapt Mistral AI models to specific needs. It details three entry points: an open-source fine-tuning SDK (mistral-finetune) built on LoRA for self-hosted fine-tuning, serverless fine-tuning services on la Plateforme leveraging proprietary techniques (also using LoRA adapters), and custom training services for enterprises using proprietary data with techniques like continuous pretraining. The post highlights the efficiency and cost-effectiveness of LoRA-based fine-tuning, showing comparable performance to full fine-tuning with reduced resource requirements.