TechPrufer
Get started
Spotify
Spotify logoSpotify · Feature trail

Software Catalog and Visualization

10 posts · Feb 2020 – Jun 2022

What this trail covers

Spotify's Backstage, a homegrown developer portal, has evolved significantly. Initially developed to address onboarding time and discoverability issues caused by rapid growth and autonomous teams, it now manages over 2,000 backend services, 300 websites, 4,000 data pipelines, and 200 mobile features. Over 200 engineers have contributed to Backstage, with 120+ plugins developed by 50+ teams, demonstrating strong internal adoption and community engagement. The platform provides a centralized location for infrastructure tooling, services, and documentation, offering a unified UI for developers to create, manage, and discover resources. This post details the open-sourcing of Backstage and its core functionalities, including creating new microservices, tracking pull requests, centralizing technical documentation, and reviewing mobile feature performance.

All Spotify trails
  1. Software Visualization — Challenge, AcceptedThis post introduces the Spotify System Model, defining core entities (APIs, Components, Resources) and higher-level abstractions (Systems, Domains) for software architecture. It details the adoption of the C4 model's notation and best practices, integrating them with the Spotify System Model to create standardized diagrams (System Landscape, System Context, System Components). The post highlights the automation of these diagrams within Backstage, ensuring they remain up-to-date with metadata and facilitating discovery, understanding of lifecycles, ownership, and relationships within the software ecosystem.
  2. How Backstage Made Our Developers More Effective — And How It Can Help Yours, TooThis post details how Spotify developed and uses Backstage to improve developer effectiveness by addressing fragmentation and discoverability issues. It highlights how Backstage serves as a central hub for finding services, managing infrastructure (like Kubernetes clusters and CI/CD pipelines), and creating new components using 'Golden Paths'. The post quantifies the impact by showing a reduction in onboarding time from over 60 days to 20 days after Backstage's introduction. It also emphasizes Backstage's role as an open-source tool with a growing plugin ecosystem.
  3. A Product Story: The Lessons of Backstage and Spotify’s Autonomous CultureThis post details the evolution of Spotify's Backstage developer portal, highlighting its origins in addressing slow onboarding and poor discoverability due to rapid growth and autonomous teams. It emphasizes how Backstage was designed to be a 'better solution, not the only solution' to align with Spotify's culture of autonomy, leading to high adoption rates (80% user satisfaction) and significant internal contributions (200+ engineers, 120+ plugins). The post also touches on the platform's role in managing a vast software ecosystem (2,000+ backend services, 4,000+ data pipelines) and its success as Spotify's largest open-source project.
  4. Designing a Better Kubernetes Experience for DevelopersThis post introduces a new Kubernetes monitoring tool as a core feature of Backstage, specifically designed for service owners. It addresses the complexity of managing services deployed across multiple Kubernetes clusters and regions by aggregating deployment status, errors, and autoscaler limits into a single, service-centric view within the Backstage developer portal. This contrasts with traditional cluster-admin-focused UIs and aims to improve developer productivity by reducing the need to navigate multiple CLIs or UIs.
  5. Happy Birthday, Backstage: Spotify’s Biggest Open Source Project Grows Up FastThis post celebrates the one-year anniversary of Backstage's open-source release. It details the project's journey from an internal tool to a CNCF Sandbox project, highlighting its adoption by companies like Expedia, Zalando, and American Airlines. Key developments in the past year include the introduction of the Service Catalog, Software Templates, TechDocs, and a Kubernetes monitoring tool. The post also announces upcoming plans to stabilize core features and APIs, standardize plugin development, and improve search and GraphQL integration. It emphasizes the project's growth in community contributions and its vision to become the standard for developer portals.
  7. Cloud Native Computing Foundation Accepts Backstage as a Sandbox ProjectThis post announces that Spotify's open-source developer portal, Backstage, has been accepted as a Sandbox project by the Cloud Native Computing Foundation (CNCF). It reiterates Backstage's core functionalities: a centralized service catalog for discoverability and control, software templates for rapid project bootstrapping, a 'docs like code' approach for technical documentation, and a plugin architecture for customization and integration. The post highlights the growing community contribution to Backstage and its potential to transform how engineers work.
  8. How We Improved Developer Productivity for Our DevOps TeamsThis post details the evolution of Spotify's developer productivity tools, focusing on the 'Golden Paths' initiative, the Tingle CI/CD system, and a test-certification program. Golden Paths, implemented via Backstage, automate project setup for various service types (backend, data pipelines, ML, web apps), reducing setup time from 14 days to under 5 minutes. Tingle provides a centralized CI/CD experience, replacing over 200 Jenkins machines and handling tens of thousands of builds weekly. The test-certification program gamifies code quality and provides insights into build times, code coverage, and test reliability, leading to a reduction in blocking bugs.
  9. How We Use Golden Paths to Solve Fragmentation in Our Software EcosystemThis post introduces and elaborates on the 'Golden Path' concept at Spotify, a strategy to combat fragmentation and 'rumour-driven development' in their software ecosystem. It details the evolution of Golden Paths from a Hack Week project for backend engineering to encompass multiple disciplines (client, data, data science, ML, web, audio processing). The post explains how Golden Paths are visualized in Backstage and documented via step-by-step tutorials using TechDocs. It highlights the success factors of these tutorials, including a defined audience, clear purpose, detailed instructions, alignment with the actual Golden Path, a per-discipline structure, an educational focus, special status in onboarding, and robust feedback mechanisms. The post also touches on future developments like an improved ownership model for documentation.
  10. How We Use Backstage at SpotifyThis post showcases how Spotify uses its internal version of Backstage, highlighting its utility in creating microservices with standardized setup (including CI and documentation), managing data pipelines with integrated tools (scheduler, log inspector, data lineage, alerts), and enabling comprehensive search for all services to improve discoverability and ownership tracking. It emphasizes the benefits of a consistent UI for better UX, a scalable plugin architecture for extensibility, and the creation of an organized ecosystem for infrastructure tools, ultimately cutting engineer onboarding time in half.
  11. What the Heck is Backstage Anyway?This post announces the open-sourcing of Backstage, Spotify's homegrown developer portal. It explains the problem of infrastructure fragmentation in growing companies and positions Backstage as a unified UI layer for all infrastructure tooling, services, and documentation. The post details four key use cases at Spotify: creating new microservices with templated best practices, following pull requests from review to production via CI integration, centralizing technical documentation using a docs-as-code approach with MkDocs, and reviewing mobile feature performance. It also outlines the vision for Backstage OSS through three phases: an extensible frontend platform, managing software inventory, and building an ecosystem of plugins.