Yahoo

Success at Apache: A Newbie’s Narrative

Feb 5, 2018
Success at Apache: A Newbie’s Narrative
SummaryAI, from the source post

This post details the author's personal journey and contributions to the Apache Tez project, focusing on the migration of Yahoo jobs from MapReduce to Tez and the significant effort to improve the Tez Shuffle Handler Service. It highlights the challenges of the last 20% of migration, the collaborative nature of open-source development, and the specific work done on progress reporting and the new Shuffle Handler implementation, which was eventually rolled out to research clusters processing a massive scale of DAGs and tasks.