Design a Double Descent Experiment
Design an Experiment Demonstrating Double Descent
This is a take-home assignment designed for a mechanistic interpretability or machine learning interview. Your primary task is to design and execute an experiment that clearly demonstrates the phenomenon of sample-to-feature-ratio double descent within a supervised learning problem. The entire submission should be achievable in approximately four hours and summarized concisely in a short slide deck.
Your submission must address the following key requirements:
- 1. Learning Setup and Dataset Generation: Select and describe a concrete supervised learning setup, including the specific model architecture and the process for generating your dataset.
- 2. Double Descent Demonstration: Systematically vary the number of training samples relative to the feature dimension. The goal is to produce a test error curve that distinctly exhibits double descent behavior, with particular attention to the region around the interpolation threshold.
- 3. Theoretical Explanation: Provide a clear theoretical explanation detailing why the double descent phenomenon occurs in your chosen setup.
- 4. Mitigation Strategy: Propose at least one method to effectively mitigate the observed double descent phenomenon. Explain the underlying reasoning for why your proposed method should be helpful.
- 5. Presentation: Present your experimental setup, the generated plots, your theoretical explanation, and the proposed mitigation strategy in a clear and concise slide deck format.
A simple high-dimensional linear regression setting is acceptable if it cleanly exhibits the effect.
AI interview prep
Core dilemma
Bridging theoretical understanding of complex ML phenomena (like double descent) with practical experimental design, empirical demonstration, and proposing effective mitigation strategies in a constrained environment.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design a Double Descent Experiment
Design an Experiment Demonstrating Double Descent
This is a take-home assignment designed for a mechanistic interpretability or machine learning interview. Your primary task is to design and execute an experiment that clearly demonstrates the phenomenon of sample-to-feature-ratio double descent within a supervised learning problem. The entire submission should be achievable in approximately four hours and summarized concisely in a short slide deck.
Your submission must address the following key requirements:
- 1. Learning Setup and Dataset Generation: Select and describe a concrete supervised learning setup, including the specific model architecture and the process for generating your dataset.
- 2. Double Descent Demonstration: Systematically vary the number of training samples relative to the feature dimension. The goal is to produce a test error curve that distinctly exhibits double descent behavior, with particular attention to the region around the interpolation threshold.
- 3. Theoretical Explanation: Provide a clear theoretical explanation detailing why the double descent phenomenon occurs in your chosen setup.
- 4. Mitigation Strategy: Propose at least one method to effectively mitigate the observed double descent phenomenon. Explain the underlying reasoning for why your proposed method should be helpful.
- 5. Presentation: Present your experimental setup, the generated plots, your theoretical explanation, and the proposed mitigation strategy in a clear and concise slide deck format.
A simple high-dimensional linear regression setting is acceptable if it cleanly exhibits the effect.