Binary Classification Via The Breast Cancer Dataset¶
The Breast Cancer Wisconsin (Diagnostic) Dataset is a widely used dataset in the field of machine learning and medical research. It contains features computed from digitized images of fine needle aspirates (FNAs) of breast masses. The dataset includes information about the characteristics of cell nuclei present in the images.
Key Features of the Dataset¶
- Number of Instances: 569
- Number of Attributes: 30 numeric features
- Target Variable: Binary classification (malignant or benign)
- Features: Measurements such as radius, texture, perimeter, area, smoothness, compactness, concavity, concave points, symmetry, and fractal dimension
Attributes Information¶
The features are computed for each cell nucleus, and the dataset includes both mean, standard error, and "worst" (mean of the three largest values) of these features.
Target Variable¶
The target variable indicates the diagnosis of the breast mass: 0 represents benign, and 1 represents malignant.
This dataset is often used for classification tasks to build models that can predict whether a breast tumor is malignant or benign based on the given features.
Example Usage in Machine Learning¶
Classification Algorithms: It can be used to train various classification algorithms including, but not limited to
logistic regression,decision trees,random forests, andsupport vector machines.Feature Selection: The dataset is also useful for performing feature selection to identify which features are most important for predicting breast cancer.
The dataset is available in the scikit-learn library, making it easy to load and use in Python for various machine learning tasks.
Model Tuner Library Instructions¶
This notebook provides a guide on how to install and use the model_tuner library in a notebook environment like Google Colab.
Model Tuner Description¶
The model_tuner library is designed to streamline the process of hyperparameter tuning and model optimization for machine learning algorithms. It provides an easy-to-use interface for defining, tuning, and evaluating models.
Key Features¶
Automatic Hyperparameter Tuning
The library can automatically tune hyperparameters for a variety of machine learning models using advanced optimization techniques.
Cross-Validation
Integrated cross-validation ensures that the models are evaluated robustly, preventing overfitting.
Documentation¶
For detailed documentation and advanced usage of the model_tuner library, please refer to the model_tuner documentation.
By following these steps, you should be able to install and use the model_tuner library effectively in your notebook environment. If you encounter any issues or have further questions, feel free to reach out for support.
Installation¶
To install the model_tuner library, use the following command:
! pip install model_tuner
Importing the Library¶
After installation, you can import the necessary components from the model_tuner library as shown below:
import model_tuner # import model_tuner to show version info.
from model_tuner import Model # Model class from model_tuner lib.
from sklearn.impute import SimpleImputer # for model imputation
from sklearn.preprocessing import StandardScaler # for feature scaling
Checking the Version¶
To ensure that the model_tuner library is installed correctly, you can check its version:
print(help(model_tuner))
Import Requisite Libraries¶
import pandas as pd
import numpy as np
from sklearn.datasets import load_breast_cancer
import xgboost as xgb
Load the dataset, define X, y¶
# Load the breast cancer dataset
data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target, name="target")
X.head() # inspect the first 5 rows of data
y.head()
if isinstance(y, pd.DataFrame):
y = y.squeeze()
Create an Instance of the XGBClassifier¶
# Creating an instance of the XGBClassifier
xgb_model = xgb.XGBClassifier(
random_state=222,
)
Define Hyperparameters for XGBoost¶
# Estimator name prefix for use in GridSearchCV or similar tools
estimator_name_xgb = "xgb"
# Define the hyperparameters for XGBoost
xgb_learning_rates = [0.1, 0.01, 0.05] # Learning rate or eta
xgb_n_estimators = [100, 200, 300] # Number of trees. Equivalent to n_estimators in GB
xgb_max_depths = [3, 5, 7] # Maximum depth of the trees
xgb_subsamples = [0.8, 1.0] # Subsample ratio of the training instances
xgb_colsample_bytree = [0.8, 1.0]
xgb_eval_metric = ["logloss"]
xgb_early_stopping_rounds = [10]
xgb_verbose = [False] # Subsample ratio of columns when constructing each tree
# Combining the hyperparameters in a dictionary
xgb_pipeline_hyperparms_grid = {
"xgb__learning_rate": xgb_learning_rates,
"xgb__n_estimators": xgb_n_estimators,
"xgb__max_depth": xgb_max_depths,
"xgb__subsample": xgb_subsamples,
"xgb__colsample_bytree": xgb_colsample_bytree,
"xgb__eval_metric": xgb_eval_metric,
"xgb__early_stopping_rounds": xgb_early_stopping_rounds,
"xgb__verbose": xgb_verbose,
"selectKBest__k": [5, 10, 20],
}
Initialize and Configure the Model¶
# Initialize ModelTuner
model_tuner = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="XGBoost_Breast_Cancer",
estimator_name=estimator_name_xgb,
calibrate=True,
estimator=xgb_model,
xgboost_early=True,
kfold=False,
selectKBest=True,
stratify_y=False,
grid=xgb_pipeline_hyperparms_grid,
# randomized_grid=True,
# n_iter=5,
scoring=["roc_auc"],
random_state=222,
n_jobs=-1,
)
Perform Grid Search Parameter Tuning¶
# Perform grid search parameter tuning
model_tuner.grid_search_param_tuning(X, y)
Fit The Model¶
# Get the training and validation data
X_train, y_train = model_tuner.get_train_data(X, y)
X_valid, y_valid = model_tuner.get_valid_data(X, y)
X_test, y_test = model_tuner.get_test_data(X, y)
# Fit the model with the validation data
model_tuner.fit(
X_train, y_train, validation_data=(X_valid, y_valid), score="roc_auc",
)
Return Metrics (Optional)¶
# Return metrics for the validation set
metrics = model_tuner.return_metrics(
X_valid,
y_valid,
)
metrics
Calibrate The Model¶
import matplotlib.pyplot as plt
from sklearn.calibration import calibration_curve
# Get the predicted probabilities for the validation data from the uncalibrated model
y_prob_uncalibrated = model_tuner.predict_proba(X_test)[:, 1]
# Compute the calibration curve for the uncalibrated model
prob_true_uncalibrated, prob_pred_uncalibrated = calibration_curve(y_test, y_prob_uncalibrated, n_bins=10)
# Calibrate the model
if model_tuner.calibrate:
model_tuner.calibrateModel(X, y, score="roc_auc")
# Predict on the validation set
y_test_pred = model_tuner.predict_proba(X_test)[:,1]
# Get the predicted probabilities for the validation data from the calibrated model
y_prob_calibrated = model_tuner.predict_proba(X_test)[:, 1]
# Compute the calibration curve for the calibrated model
prob_true_calibrated, prob_pred_calibrated = calibration_curve(y_test, y_prob_calibrated, n_bins=10,)
# Plot the calibration curves
plt.figure(figsize=(5, 5))
plt.plot(prob_pred_uncalibrated, prob_true_uncalibrated, marker='o', label='Uncalibrated XGBoost')
plt.plot(prob_pred_calibrated, prob_true_calibrated, marker='o', label='Calibrated XGBoost')
plt.plot([0, 1], [0, 1], linestyle='--', label='Perfectly calibrated')
plt.xlabel('Predicted probability')
plt.ylabel('True probability in each bin')
plt.title('Calibration plot (reliability curve)')
plt.legend()
plt.show()
Classification Report¶
print(model_tuner.classification_report)
Reference¶
Scikit-learn Developers. (2023). Breast Cancer Wisconsin (Diagnostic) Dataset. Scikit-learn Documentation.
https://scikit-learn.org/stable/datasets/toy_dataset.html#breast-cancer-wisconsin-dataset.