End-to-End Signup Prediction with scikit-learn

Context

You are provided with a cleaned tabular dataset containing marketing and product metrics. Your objective is to develop a robust, leakage-free modeling workflow using scikit-learn to predict daily signups.

Assume the dataset is a pandas DataFrame that includes at least the following columns:

• spend (float)
• clicks (float)
• cpc (float)
• region (categorical string)
• date (datetime or string parsable to datetime)
• signups (non-negative integer/float target)

Task

Build a machine learning model to predict signups using spend, clicks, cpc, region, and a time trend derived from the date column. Implement the following steps within scikit-learn:

1. ColumnTransformer Setup
   • Numeric features: spend, clicks, cpc, and a time_index (extracted from date). Apply PolynomialFeatures(degree=2, include_bias=False) followed by StandardScaler().
   • Categorical feature: region. Apply OneHotEncoder(handle_unknown='ignore').
2. Target Transformation Decision
   • Perform a quick diagnostic (e.g., via cross-validated RMSE comparison on a simple baseline model) to determine whether to apply log1p to the target variable. If log1p is selected, use TransformedTargetRegressor.
3. Model Selection and Tuning
   • Construct a Pipeline that encapsulates all preprocessing steps and a linear penalized model.
   • Tune Ridge, Lasso, and ElasticNet models using GridSearchCV with 5-fold cross-validation by default.
   • If temporal dependence is detected in the data, switch both the inner and outer cross-validation strategies to TimeSeriesSplit.
4. Evaluation and Leakage Control
   • Create a held-out test split before any cross-validation or preprocessing. Use a chronological split if temporal dependence is detected; otherwise, use a random 80/20 split.
   • Ensure that all preprocessing steps (including the creation of time_index) are strictly contained within the Pipeline to prevent data leakage.
   • Report RMSE as the primary evaluation metric, and also report R^2 on the held-out test set.
   • Utilize nested cross-validation to obtain a robust estimate of the model's generalization RMSE.
5. Feature Effects Analysis
   • After selecting the best model, extract and rank the top 10 features based on their absolute standardized effect size.
   • Map one-hot encoded feature names back to a human-readable format (e.g., region=NA, clear interaction terms).
6. Model Persistence
   • Provide code to save the trained model to disk and demonstrate how to reload it and use it to score new data.
7. Rationale
   • Offer a concise justification for each significant modeling choice made throughout the process.