TechPrufer
Get started
Atlassian
Atlassian logoAtlassian · Feature trail

Jira Cloud Data Export Service

2 posts · Feb 2025 – Apr 2025

What this trail covers

The Jira Cloud Data Export Service has evolved to improve export throughput and handle Jira's data constraints. The service now employs a hybrid architecture, leveraging Atlassian's Lithium streaming platform for high-volume entities (like issues, comments, and change history) and HTTP Batch APIs for low-volume, complex entities. This evolution includes optimizing extraction by splitting tasks into parallel jobs executed by multiple threads and compute nodes, leading to a 5x improvement in throughput. This post details the architectural evolution and engineering enhancements of a new external migration platform, which has significantly optimized the migration process for Jira users. With this new platform, migrations are now 20 times faster and achieve a data throughput that is 30 times better than the legacy data importer. Furthermore, a 95% improvement in the reliability rate of migrations has been realized.

All Atlassian trails
  1. Boosting Performance: How We Scaled and Enhanced Throughput in the Jira Export ServiceThis post details the evolution of the Jira Cloud Data Export Service, focusing on architectural changes and code optimizations to improve export throughput. Key contributions include: adopting a hybrid architecture leveraging Lithium streaming for high-volume entities (issues, comments, change history) and HTTP Batch APIs for low-volume entities; optimizing high-volume entity extraction by splitting tasks into parallel jobs executed by multiple threads and compute nodes; and addressing N+1 query issues in key export code paths. The MVP design achieved a 5x throughput improvement, and the current design further scales horizontally by engaging multiple compute nodes based on project data size.
  2. Scaling Jira Cloud migrations with external importersThis post details the architectural evolution and engineering enhancements of a new external migration platform for Jira Cloud. It highlights the transition from a legacy, monolithic importer with scalability and reliability issues (6,000 issues/hour, 70-80% success rate) to a new microservices-based platform. The new platform achieves 20x faster migrations and 30x better throughput, with a 95% reliability rate. Phase 1 focused on building scalable CSV importers using a microservices architecture with components like a Migration Service, Migration Process Orchestrator (leveraging AWS Step Functions and SQS), Migration Transformer Engine, User Migration Service, and Project Import Service. Key benefits include optimized scalability, performance, reliability through asynchronous processing and intelligent retries, and reduced technical debt.