Table of Contents

Getting Started

• Welcome to the EquiBoots Documentation!
  • Project Links
  • Prerequisites
  • Installation
  • Description

Point Estimate Metrics

• Point Estimate Evaluation
  • Supported Metrics
  • Initial Set-up
    • Step 1: Import and Initialize EquiBoots
    • Step 2: Slice Groups and Compute Point Estimates
  • Metrics DataFrame
    • metrics_dataframe()
  • Statistical Tests
    • analyze_statistical_significance()
    • Example
  • Statistical Significance Plots
    • Test Setup
• Group Metrics Point Plot
  • eq_group_metrics_point_plot()
  • Statistical Metrics table
    • metrics_table()
• Group Curve Plots
  • eq_plot_group_curves()
  • ROC AUC Curve
  • Precision-Recall Curves
  • Calibration Plots
    • Example 1 (Calibration Overlay)
    • Example 2 (Calibration Subplots)
    • Example 3 (LOWESS Calibration)
    • Example 4 (Calibration Histograms)
• Group Metrics Forest Plot
  • eq_plot_metrics_forest()
  • Example 1: Prevalance
• Effect Size
  • plot_effect_sizes()
• Residual-Based Point Estimates
  • eq_plot_residuals_by_group()
  • Example 1: Residuals by Age Group (Overlayed)
  • Example 2: Residuals by Age Group (Subplots)

Bootstrap Estimate Metrics

• Bootstrap Estimate Evaluation
  • 1. Bootstrap Setup
    • Step 1.1: Instantiate EquiBoots with Bootstrapping
    • Step 1.2: Slice by Group and Compute Metrics
  • 2. Disparity Analysis
    • Plot Disparity Ratios
      • eq_group_metrics_plot()
  • 3. Metric Differences
  • 4. Statistical Significance Testing
    • 4.1: Metrics Table with Significance Annotations
    • 4.2: Visualize Differences with Significance
  • Bootstrapped Group Curve Plots
    • eq_plot_bootstrapped_group_curves()
    • ROC AUC Curves
    • Precision-Recall Curves
    • Calibration Curves
  • Bootstrapped Forest Plots
    • eq_plot_bootstrap_forest()
    • Tabular Breakdown of Bootstrapped Metrics
      • calculate_bootstrap_stats()
  • Summary

Classes, Attributes, & Methods

• EquiBoots Class
  • EquiBoots
    • EquiBoots.calculate_differences()
    • EquiBoots.calculate_disparities()
    • EquiBoots.calculate_groups_differences()
    • EquiBoots.calculate_groups_disparities()
    • EquiBoots.check_classification_task()
    • EquiBoots.check_fairness_vars()
    • EquiBoots.check_group_empty()
    • EquiBoots.check_group_size()
    • EquiBoots.check_task()
    • EquiBoots.get_groups_metrics()
    • EquiBoots.get_metrics()
    • EquiBoots.list_adjustment_methods()
    • EquiBoots.list_available_tests()
    • EquiBoots.set_fix_seeds()
    • EquiBoots.set_reference_groups()
  • Overview
  • Constructor
  • Main Methods
    • grouper()
    • slicer()
    • get_metrics()
    • calculate_disparities()
    • calculate_differences()
    • analyze_statistical_significance()
    • set_fix_seeds()
    • list_available_tests()
    • list_adjustment_methods()
  • Non-Main/Internal Methods
    • set_reference_groups()
    • check_task()
    • check_classification_task()
    • check_fairness_vars()
    • check_group_size()
    • check_group_empty()
    • sample_group()
    • groups_slicer()
    • get_groups_metrics()
    • calculate_groups_disparities()
    • calculate_groups_differences()
  • Example Usage
  • Point Estimates Without Bootstrapping
  • With Bootstrapping
• StatisticalTester
  • Overview
  • Classes
    • StatTestResult
      • StatTestResult
    • StatisticalTester
      • StatisticalTester
  • Function Signatures
    • StatTestResult
    • StatisticalTester
      • StatisticalTester._bootstrap_test()
      • StatisticalTester.get_ci_bounds()
      • StatisticalTester.calc_p_value_bootstrap()
      • StatisticalTester._chi_square_test()
      • StatisticalTester._calculate_effect_size()
      • StatisticalTester._adjust_p_values()
      • StatisticalTester.analyze_metrics()
      • StatisticalTester.adjusting_p_vals()
      • StatisticalTester._validate_config()
      • StatisticalTester.cohens_d()
      • StatisticalTester._analyze_single_metrics()
      • StatisticalTester._analyze_bootstrapped_metrics()
  • Usage Example

From Model to Prediction

• Preparing Model Outputs for Fairness Analysis
  • Step 1: Install and Import Dependencies
  • Step 2: Load the Dataset
  • Step 3: Clean the Data
  • Step 4: Encode the Target Variable
  • Step 5: Prepare Features and Labels
  • Step 6: Train-Test Split
  • Step 7: Train the XGBoost Model
  • Step 8: Generate Predictions and Evaluate
  • Step 9: Extract Model Outputs

Mathematical Framework

• Calibration Curves and Area Under the Curve
  • Calibration Curves and Area Interpretation
    • Example 1: Calibration with y = x²
    • Example 2: Calibration with y = x² + 4x
• Regression Residuals

iPython Notebooks

• Problem Types
  • Binary Classification
  • Regression

About EquiBoots

• Acknowledgements
• Contributors
• Citing EquiBoots
• Changelog
  • Version 0.0.1a1
  • Version 0.0.1a
  • Version 0.0.0a10
    • Add histogram-only mode to calibration curves (plot_hist flag)
    • Handle Seaborn < 0.13.2 legend errors for boxplot/violinplot
    • Add lowess_kwargs support & show LOWESS AUC in legend
  • Version 0.0.0a9
  • What’s Changed
  • Version 0.0.0a8
  • Version 0.0.0a7
  • Version 0.0.0a6
  • Version 0.0.0a5
    • New Contributor
  • Version 0.0.0a4
    • New Contributor
  • Version 0.0.0a3
  • Version 0.0.0a2
  • Version 0.0.0a1
    • New Contributors
• References