Implement blocked time-series cross-validation
Problem Description
Design and implement a panel-aware blocked time-series cross-validation splitter that incorporates an embargo period. The input is a DataFrame containing columns loan_id, msa, and month.
Requirements
- Expanding-Window Folds with Embargo:
- Generate
K=5expanding-window folds with defined cutoffst1 < ... < t5. - For each fold
k:- The training set must include data from months less than or equal to
tk - embargo. - The test set must include data from months strictly greater than
tk - embargoup totk+1. - An
embargoperiod of 90 days must be enforced. This means noloan_idcan appear in both the training and test sets within the(tk - embargo, tk+1]window.
- The training set must include data from months less than or equal to
- Generate
- Grouped Blocking:
- The splitter must support grouped blocking. This implies that for at least one fold, all months associated with an entire MSA (Metropolitan Statistical Area) must be held out exclusively for the test set.
- Temporal order within each MSA must be preserved in both training and test sets.
- Determinism and Stability:
- The splitter's output must be deterministic and stable, meaning it should produce identical splits regardless of the input DataFrame's row order (e.g., shuffled input).
- Algorithm Analysis:
- Provide precise algorithmic steps or pseudocode for the splitter.
- Analyze the time complexity in terms of
N(number of rows),G(number of MSAs), andK(number of folds). - Discuss the memory characteristics of the proposed solution.
- Unit Testing:
- Describe specific unit tests for adversarial cases, including:
- Duplicate timestamps.
- Missing months in the time series.
- Scenarios where a
loan_idmigrates across different MSAs. - Highly imbalanced MSAs (e.g., one MSA with vast majority of data).
- Describe specific unit tests for adversarial cases, including:
AI interview prep
Core dilemma
The central challenge is designing a robust, efficient, and correct data partitioning strategy that simultaneously satisfies multiple, potentially conflicting, constraints: temporal ordering, preventing data leakage (embargo), and ensuring group integrity (MSA blocking), all while maintaining determinism and scalability.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Implement blocked time-series cross-validation
Problem Description
Design and implement a panel-aware blocked time-series cross-validation splitter that incorporates an embargo period. The input is a DataFrame containing columns loan_id, msa, and month.
Requirements
- Expanding-Window Folds with Embargo:
- Generate
K=5expanding-window folds with defined cutoffst1 < ... < t5. - For each fold
k:- The training set must include data from months less than or equal to
tk - embargo. - The test set must include data from months strictly greater than
tk - embargoup totk+1. - An
embargoperiod of 90 days must be enforced. This means noloan_idcan appear in both the training and test sets within the(tk - embargo, tk+1]window.
- The training set must include data from months less than or equal to
- Generate
- Grouped Blocking:
- The splitter must support grouped blocking. This implies that for at least one fold, all months associated with an entire MSA (Metropolitan Statistical Area) must be held out exclusively for the test set.
- Temporal order within each MSA must be preserved in both training and test sets.
- Determinism and Stability:
- The splitter's output must be deterministic and stable, meaning it should produce identical splits regardless of the input DataFrame's row order (e.g., shuffled input).
- Algorithm Analysis:
- Provide precise algorithmic steps or pseudocode for the splitter.
- Analyze the time complexity in terms of
N(number of rows),G(number of MSAs), andK(number of folds). - Discuss the memory characteristics of the proposed solution.
- Unit Testing:
- Describe specific unit tests for adversarial cases, including:
- Duplicate timestamps.
- Missing months in the time series.
- Scenarios where a
loan_idmigrates across different MSAs. - Highly imbalanced MSAs (e.g., one MSA with vast majority of data).
- Describe specific unit tests for adversarial cases, including: