Baseten

New observability features: activity logging, LLM metrics, and metrics dashboard customization

Dec 20, 2024
New observability features: activity logging, LLM metrics, and metrics dashboard customization
SummaryAI, from the source post

This post introduces three new observability features: activity logging for timestamped changes to workspace and models, request-level LLM metrics (request size, response size, time to first byte) to aid in debugging latency spikes, and customizable metrics dashboards for unified views, reordering, and direct log access from metric plots. These features aim to improve monitoring, debugging, and optimization of production AI workloads.

More from the Baseten AI Model Deployment and Serving Platform trail