Spotify

How to Accurately Test Significance with Difference in Difference Models

Sep 1, 2023Bradley Doll (Insights Manager), Jeffrey Li (Insights Manager), and Tolga Oztan (Sr. Data Scientist)
How to Accurately Test Significance with Difference in Difference Models
SummaryAI, from the source post

This post introduces and evaluates three methods (averaging, clustering, and permutation testing) for accurately estimating significance in difference-in-differences (DID) models when dealing with autocorrelated time series data. It highlights the limitations of standard OLS significance tests in such scenarios and provides simulation-based evidence on the trade-offs between power and false positives for each method, recommending permutation testing for smaller datasets and clustered standard errors for larger ones.

More from the Experimentation Platform trail