Measuring time savings from Figma Make

What this post added

This post details the methodology and findings of a randomized controlled trial (RCT) designed to measure the time savings and efficiency gains from using Figma Make. It explains the challenges of measuring AI impact due to confounders and why traditional methods like A/B testing and causal inference were insufficient. The post outlines the RCT design, including participant selection (100 participants: 50 product designers, 50 product managers), task design (social media post editing, dark mode conversion, adding menu options, building comment fly-up interactions), and moderation procedures. Key results highlight overall time savings (20%), ease improvement (16%), and usability gains (15%), with specific benefits for PMs and differentiated impacts on designers based on task complexity.