Model Tuner Documentation

Getting Started

• Welcome to Model Tuner’s Documentation!
  • What Does Model Tuner Offer?
  • Prerequisites
  • Installation

Usage Guide

• iPython Notebooks
  • Binary Classification Examples
  • Regression Example
• Key Methods and Functionalities
• Helper Functions
• Input Parameters
  • Model
• Pipeline Management
  • Specifying Pipeline Steps
  • Helper Methods for Pipeline Extraction
    • get_preprocessing_and_feature_selection_pipeline()
    • get_feature_selection_pipeline()
    • get_preprocessing_pipeline()
  • Summary
• Binary Classification
  • AIDS Clinical Trials Group Study
    • Step 1: Import Necessary Libraries
    • Step 2: Load the dataset, define X, y
    • Step 3: Check for zero-variance columns and drop accordingly
    • Step 4: Create an Instance of the XGBClassifier
    • Step 5: Define Hyperparameters for XGBoost
    • Step 6: Initialize and Configure the Model
    • Step 7: Perform Grid Search Parameter Tuning
    • Step 8: Fit the Model
    • Step 9: Return Metrics (Optional)
    • Step 10: Calibrate the Model (if needed)
    • Classification Report (Optional)
  • Recursive Feature Elimination (RFE)
    • Elastic Net for Feature Selection with RFE
  • Imbalanced Learning
    • Generating an Imbalanced Dataset
    • Define Hyperparameters for XGBoost
    • Define The Model object
    • Addressing Class Imbalance in Machine Learning
      • Techniques to Address Class Imbalance
      • Purpose of Using These Techniques
    • Synthetic Minority Oversampling Technique (SMOTE)
      • Initalize and Configure The Model
      • Perform Grid Search Parameter Tuning and Retrieve Split Data
      • SMOTE: Distribution of y values after resampling
      • Fit The Model
      • Return Metrics (Optional)
  • SHAP (SHapley Additive exPlanations)
    • Step 1: Transform the test data using the feature selection pipeline
    • Step 2: Retrieve the trained XGBoost classifier from the pipeline
    • Step 3: Extract feature names from the training data, and initialize the SHAP explainer for the XGBoost classifier
    • Step 4: Compute SHAP values for the transformed test dataset and generate a summary plot of SHAP values
    • Step 5: Generate a summary plot of SHAP values
    • Feature Importance and Impact
• Regression
  • California Housing with XGBoost
    • Step 1: Import Necessary Libraries
    • Step 2: Load the Dataset
    • Step 3: Create an Instance of the XGBRegressor
    • Step 4: Define Hyperparameters for XGBoost
    • Step 5: Initialize and Configure the Model
    • Step 6: Perform Grid Search Parameter Tuning and Retrieve Split Data
    • Step 7: Fit the Model
    • Step 8: Return Metrics (Optional)
• Bootstrap Metrics
  • check_input_type()
  • sampling_method()
  • evaluate_bootstrap_metrics()
  • return_bootstrap_metrics()
  • Bootstrap Metrics Example

Caveats

• Zero Variance Columns
  • Effects on Model Training
• Dependent Variable
  • Target Variable Shape and Its Effects
  • Solution
• Imputation Before Scaling
  • 1. Accurate Calculation of Scaling Parameters
  • 2. Consistency in Data Transformation
  • 3. Prevention of Distortion in Scaling
• Column Stratification with Cross-Validation
  • Cross-Validation and Stratification
• Model Calibration
  • Goal of Calibration
  • Calibration Curve
  • Brier Score
  • Platt Scaling
  • Isotonic Regression
  • Example: Calibration in Logistic Regression
  • Summary
• Using Imputation and Scaling in Pipeline Steps for Model Preprocessing
  • When Is Imputation and Feature Scaling in pipeline_steps Beneficial?
  • Models Not Benefiting From Imputation and Scaling in pipeline_steps
  • Why Doesn’t XGBoost Require Imputation and Scaling in pipeline_steps?
• Caveats in Imbalanced Learning
  • Bias from Class Distribution
  • Threshold-Dependent Predictions
  • Limitations of Accuracy
    • Integration and Practical Considerations
  • Impact of Resampling Techniques
    • SMOTE: A Mathematical Illustration
    • Example of Synthetic Sample Creation
    • Mitigating the Caveats
• ElasticNet Regularization
  • Important Considerations

About Model Tuner

• GitHub Repository
• Acknowledgements
• Citing Model Tuner
• Changelog
  • Version 0.0.22a
  • Version 0.0.21a
  • Version 0.0.20a
  • Version 0.0.19a
  • Version 0.0.18a
  • Version 0.0.17a
  • Version 0.0.16a
  • Version 0.0.15a
  • Version 0.0.014a
  • Version 0.0.013a
  • Version 0.0.012a
  • Version 0.0.011a
  • Version 0.0.010a
  • Version 0.0.09a
  • Version 0.0.08a
  • Version 0.0.07a
  • Version 0.0.06a
  • Version 0.0.05a
  • Version 0.0.02a
• References