Debug MNIST denoiser training
Debugging a Denoising Network on MNIST
Your goal is to create a Colab notebook that successfully trains a denoising neural network on the MNIST dataset, ensuring two key outcomes:
- (a) The training loss steadily decreases.
- (b) The evaluation loss remains close to the training loss.
To achieve this, you must identify and rectify specific issues across three critical areas, providing a clear description of how you validate each fix.
1. Data
- Train/Test Distribution Mismatch: Detect and correct an issue where the test set does not adequately cover all digits (0–9). Ensure the corrected test set is representative.
- Normalization Consistency: Verify and enforce that the same normalization statistics (mean and standard deviation), computed exclusively from the training set, are applied uniformly to both the training and test data.
2. Model
- Inappropriate Final Activation: Identify and either remove or replace an unsuitable ReLU activation function in the model's final layer. The output layer must be capable of producing negative values, which are typical for denoised signals.
3. Optimization
- Missing
optimizer.zero_grad(): Add the necessaryoptimizer.zero_grad()call to prevent gradient accumulation across batches. - Incorrect Optimization Order: Correct the sequence of
backward()andoptimizer.step()calls to ensure gradients are computed and applied in the proper order.
For each of these three sections (Data, Model, Optimization), you are required to describe:
- The sanity checks and assertions you would integrate into the code.
- The minimal code changes and key snippets required to implement the fix.
- The specific metrics you would track to validate the effectiveness of your fix.
AI interview prep
Core dilemma
Systematic diagnosis and remediation of common machine learning training pipeline failures, ensuring both convergence and generalization through rigorous validation and understanding of fundamental ML principles.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Debug MNIST denoiser training
Debugging a Denoising Network on MNIST
Your goal is to create a Colab notebook that successfully trains a denoising neural network on the MNIST dataset, ensuring two key outcomes:
- (a) The training loss steadily decreases.
- (b) The evaluation loss remains close to the training loss.
To achieve this, you must identify and rectify specific issues across three critical areas, providing a clear description of how you validate each fix.
1. Data
- Train/Test Distribution Mismatch: Detect and correct an issue where the test set does not adequately cover all digits (0–9). Ensure the corrected test set is representative.
- Normalization Consistency: Verify and enforce that the same normalization statistics (mean and standard deviation), computed exclusively from the training set, are applied uniformly to both the training and test data.
2. Model
- Inappropriate Final Activation: Identify and either remove or replace an unsuitable ReLU activation function in the model's final layer. The output layer must be capable of producing negative values, which are typical for denoised signals.
3. Optimization
- Missing
optimizer.zero_grad(): Add the necessaryoptimizer.zero_grad()call to prevent gradient accumulation across batches. - Incorrect Optimization Order: Correct the sequence of
backward()andoptimizer.step()calls to ensure gradients are computed and applied in the proper order.
For each of these three sections (Data, Model, Optimization), you are required to describe:
- The sanity checks and assertions you would integrate into the code.
- The minimal code changes and key snippets required to implement the fix.
- The specific metrics you would track to validate the effectiveness of your fix.