UCI Machine Learning Repository Dataset - AIDS Clinical Trials Group Study 175¶
The UCI Machine Learning Repository is a well-known resource for accessing a wide range of datasets used for machine learning research and practice. One such dataset is the AIDS Clinical Trials Group Study dataset, which can be used to build and evaluate predictive models.
In our library, you can easily fetch this dataset using the ucimlrepo package. If you haven't installed it yet, you can do so by running pip install ucimlrepo.
Model Tuner Library Instructions¶
This notebook provides a guide on how to install and use the model_tuner library in a notebook environment like Google Colab.
Model Tuner Description¶
The model_tuner library is designed to streamline the process of hyperparameter tuning and model optimization for machine learning algorithms. It provides an easy-to-use interface for defining, tuning, and evaluating models.
Key Features¶
Automatic Hyperparameter Tuning
The library can automatically tune hyperparameters for a variety of machine learning models using advanced optimization techniques.
Cross-Validation
Integrated cross-validation ensures that the models are evaluated robustly, preventing overfitting.
Documentation¶
For detailed documentation and advanced usage of the model_tuner library, please refer to the model_tuner documentation.
By following these steps, you should be able to install and use the model_tuner library effectively in your notebook environment. If you encounter any issues or have further questions, feel free to reach out for support.
Installation¶
To install the model_tuner library, use the following command:
! pip install model_tuner
Importing the Library¶
After installation, you can import the necessary components from the model_tuner library as shown below:
import model_tuner # import model_tuner to show version info.
from model_tuner import Model # Model class from model_tuner lib.
from sklearn.impute import SimpleImputer # for model imputation
Checking the Version¶
To ensure that the model_tuner library is installed correctly, you can check its version:
print(help(model_tuner))
Binary Classification Via The Breast Cancer Dataset¶
AIDS Clinical Trials Group Study 175 Dataset¶
The AIDS Clinical Trials Group Study 175 Dataset is a healthcare dataset that contains statistical and categorical information about patients who have been diagnosed with AIDS. This dataset, which was initially published in 1996, is often used to predict whether or not a patient will respond to different AIDS treatments.
Key Features of the Dataset¶
- Number of Instances: 2,139
- Number of Features: 23
- Feature Type: Categorical, Integer
- Subject Area: Health and Medicine
- Associated Tasks: Classification, Regression
Dataset Information¶
- Purpose of the Dataset: The dataset was created to examine the performance of two different types of AIDS treatments.
- Funding: The creation of this dataset was funded by the AIDS Clinical Trials Group of the National Institute of Allergy and Infectious Diseases and General Research Center units funded by the National Center for Research Resources.
- Instances Represent: The dataset includes health records of AIDS patients from the US only. Sensitive Data: The dataset includes sensitive information such as ethnicity (race) and gender.
- Data Preprocessing: No preprocessing was performed on the data.
- Missing Values: The dataset does not have missing values.
Example Usage in Machine Learning¶
- Predictive Modeling: The dataset can be used to train models that predict patient outcomes based on demographic and clinical features.
- Treatment Efficacy Analysis: Researchers can use the dataset to compare the effectiveness of different AIDS treatments.
- Health Data Analytics: This dataset is valuable for analyzing trends in the progression and treatment of AIDS among patients in the United States.
Accessing the Dataset¶
To work with the AIDS Clinical Trials Group Study 175 Dataset, you can load it using the ucimlrepo package. If you haven't installed it yet, install it with:
pip install ucimlrepo
! pip install ucimlrepo
Load the dataset, define X, y¶
Once installed, you can quickly load the AIDS Clinical Trials Group Study dataset with a few simple commands:
from ucimlrepo import fetch_ucirepo
# fetch dataset
aids_clinical_trials_group_study_175 = fetch_ucirepo(id=890)
# data (as pandas dataframes)
X = aids_clinical_trials_group_study_175.data.features
y = aids_clinical_trials_group_study_175.data.targets
Import Requisite Libraries¶
import pandas as pd
import numpy as np
import xgboost as xgb
from sklearn.datasets import load_diabetes
X.head() # inspect the first 5 rows of data
if isinstance(y, pd.DataFrame):
y = y.squeeze()
Check for zero-variance columns and drop accordingly¶
# Check for zero-variance columns and drop them
zero_variance_columns = X.columns[X.var() == 0]
if not zero_variance_columns.empty:
X = X.drop(columns=zero_variance_columns)
Create an Instance of the XGBClassifier¶
# Creating an instance of the XGBClassifier
xgb_model = xgb.XGBClassifier(
random_state=222,
)
Define Hyperparameters for XGBoost¶
# Estimator name prefix for use in GridSearchCV or similar tools
estimator_name_xgb = "xgb"
# Define the hyperparameters for XGBoost
xgb_learning_rates = [0.1, 0.01, 0.05] # Learning rate or eta
xgb_n_estimators = [100, 200, 300] # Number of trees. Equivalent to n_estimators in GB
xgb_max_depths = [3, 5, 7] # Maximum depth of the trees
xgb_subsamples = [0.8, 1.0] # Subsample ratio of the training instances
xgb_colsample_bytree = [0.8, 1.0]
xgb_eval_metric = ["logloss"]
xgb_early_stopping_rounds = [10]
xgb_verbose = [False] # Subsample ratio of columns when constructing each tree
# Combining the hyperparameters in a dictionary
xgb_pipeline_hyperparms_grid = {
"xgb__learning_rate": xgb_learning_rates,
"xgb__n_estimators": xgb_n_estimators,
"xgb__max_depth": xgb_max_depths,
"xgb__subsample": xgb_subsamples,
"xgb__colsample_bytree": xgb_colsample_bytree,
"xgb__eval_metric": xgb_eval_metric,
"xgb__early_stopping_rounds": xgb_early_stopping_rounds,
"xgb__verbose": xgb_verbose,
"selectKBest__k": [5,10,20],
}
Initialize and Configure the Model¶
from model_tuner import Model
# Initialize ModelTuner
model_tuner = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="XGBoost_AIDS",
estimator_name=estimator_name_xgb,
calibrate=True,
estimator=xgb_model,
xgboost_early=True,
kfold=False,
selectKBest=True,
stratify_y=True,
stratify_cols=["gender", "race"],
grid=xgb_pipeline_hyperparms_grid,
# randomized_grid=True,
# n_iter=5,
scoring=["roc_auc"],
random_state=222,
n_jobs=-1,
)
Perform Grid Search Parameter Tuning¶
# Perform grid search parameter tuning
model_tuner.grid_search_param_tuning(X,y)
Fit the Model¶
# Get the training and validation data
X_train, y_train = model_tuner.get_train_data(X, y)
X_valid, y_valid = model_tuner.get_valid_data(X, y)
X_test, y_test = model_tuner.get_test_data(X, y)
# Fit the model with the validation data
model_tuner.fit(
X_train,
y_train,
validation_data=(X_valid, y_valid),
score="roc_auc",
)
Return Metrics (Optional)¶
# Return metrics for the validation set
metrics = model_tuner.return_metrics(
X_valid,
y_valid,
)
metrics
Calibrate the Model¶
import matplotlib.pyplot as plt
from sklearn.calibration import calibration_curve
# Get the predicted probabilities for the validation data from the uncalibrated model
y_prob_uncalibrated = model_tuner.predict_proba(X_test)[:, 1]
# Compute the calibration curve for the uncalibrated model
prob_true_uncalibrated, prob_pred_uncalibrated = calibration_curve(
y_test,
y_prob_uncalibrated,
n_bins=10,
)
# Calibrate the model
if model_tuner.calibrate:
model_tuner.calibrateModel(X, y, score="roc_auc")
# Predict on the validation set
y_test_pred = model_tuner.predict_proba(X_test)[:,1]
# Get the predicted probabilities for the validation data from calibrated model
y_prob_calibrated = model_tuner.predict_proba(X_test)[:, 1]
# Compute the calibration curve for the calibrated model
prob_true_calibrated, prob_pred_calibrated = calibration_curve(
y_test,
y_prob_calibrated,
n_bins=5,
)
# Plot the calibration curves
plt.figure(figsize=(5, 5))
plt.plot(
prob_pred_uncalibrated,
prob_true_uncalibrated,
marker="o",
label="Uncalibrated XGBoost",
)
plt.plot(
prob_pred_calibrated,
prob_true_calibrated,
marker="o",
label="Calibrated XGBoost",
)
plt.plot(
[0, 1],
[0, 1],
linestyle="--",
label="Perfectly calibrated",
)
plt.xlabel("Predicted probability")
plt.ylabel("True probability in each bin")
plt.title("Calibration plot (reliability curve)")
plt.legend()
plt.show()
print(model_tuner.classification_report)
Reference¶
El-Sadr, W., & Abrams, D. (1998). AIDS Clinical Trials Group Study 175. UCI Machine Learning Repository.
https://doi.org/10.24432/C5G896.