This Google Colab notebook utilizes a dataset on the Los Angeles Real Estate market sourced from Redfin, consisting of 200 rows and 27 columns. The dataset captures a snapshot in time of various property listings, providing detailed information about each property.
The dataset includes the following columns:
SALE TYPESOLD DATEPROPERTY TYPEADDRESSCITYSTATE OR PROVINCEZIP OR POSTAL CODEPRICEBEDSBATHSLOCATIONSQUARE FEETLOT SIZEYEAR BUILTDAYS ON MARKET$/SQUARE FEETHOA/MONTHSTATUSNEXT OPEN HOUSE START TIMENEXT OPEN HOUSE END TIMEURL (SEE https://www.redfin.com/buy-a-home/comparative-market-analysis FOR INFO ON PRICING)SOURCEMLS#FAVORITEINTERESTEDLATITUDELONGITUDEThe primary purpose of this notebook is to demonstrate various regression examples using the Redfin dataset. Specifically, the following variables are used for testing in this notebook:
Features (X):
BEDSBATHSSQUARE FEETLOT SIZETarget (y):
PRICEThe dataset is sourced from Redfin, available at Redfin's Comparative Market Analysis.
This dataset represents a single snapshot in time and may not reflect current market conditions.
This notebook provides a guide on how to install and use the model_tuner library in a notebook environment like Google Colab.
The model_tuner library is designed to streamline the process of hyperparameter tuning and model optimization for machine learning algorithms. It provides an easy-to-use interface for defining, tuning, and evaluating models.
Automatic Hyperparameter Tuning
The library can automatically tune hyperparameters for a variety of machine learning models using advanced optimization techniques.
Cross-Validation
Integrated cross-validation ensures that the models are evaluated robustly, preventing overfitting.
For detailed documentation and advanced usage of the model_tuner library, please refer to the model_tuner documentation.
By following these steps, you should be able to install and use the model_tuner library effectively in your notebook environment. If you encounter any issues or have further questions, feel free to reach out for support.
To install the model_tuner library, use the following command:
! pip install model_tuner
After installation, you can import the necessary components from the model_tuner library as shown below:
import model_tuner
from model_tuner import Model
from sklearn.impute import SimpleImputer
To ensure that the model_tuner library is installed correctly, you can check its version:
print(help(model_tuner))
import pandas as pd
import numpy as np
from sklearn.linear_model import Lasso, Ridge, SGDRegressor
from xgboost import XGBRegressor
import warnings
from sklearn.exceptions import DataConversionWarning
warnings.filterwarnings(action="ignore", category=DataConversionWarning)
warnings.simplefilter(action='ignore', category=UserWarning)
import pandas as pd
import numpy as np
from sklearn.datasets import load_breast_cancer
import xgboost as xgb
# Direct download link to the Excel file
url = (
"https://github.com/uclamii/model_tuner/raw/main/public_data/"
"redfin_2024-04-16-15-59-17.xlsx"
)
# Read the Excel file
df = pd.read_excel(url)
df.head() # inspect first 5 rows of data
df.columns # inspect the list of cols in the dataset
print(f"This dataset has {df.shape[0]} rows and {df.shape[1]} columns.")
df = df.drop(df.index[0]) # remove first row of dataframe which is not used
X = df[["BEDS", "BATHS", "SQUARE FEET", "LOT SIZE"]]
y = df[["PRICE"]]
# Define the set of hyperparameters to tune
lasso_pipeline_hyperparms_grid = [
{
"lasso__fit_intercept": [True, False],
"lasso__precompute": [False],
"lasso__copy_X": [True, False],
"lasso__max_iter": [100, 500, 1000, 2000],
"lasso__tol": [1e-4, 1e-3],
"lasso__warm_start": [True, False],
"lasso__positive": [True, False],
}
]
lasso_reg = Lasso()
estimator_name = "lasso"
# Set the parameters by cross-validation
kfold = False
calibrate = False
Model¶redfin_lasso = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="Redfin_model_Lasso",
estimator_name=estimator_name,
model_type="regression",
calibrate=calibrate,
estimator=lasso_reg,
kfold=kfold,
stratify_y=False,
grid=lasso_pipeline_hyperparms_grid,
randomized_grid=True,
scoring=["r2"],
random_state=3,
)
redfin_lasso.grid_search_param_tuning(X, y)
X_train, y_train = redfin_lasso.get_train_data(X, y)
X_test, y_test = redfin_lasso.get_test_data(X, y)
X_valid, y_valid = redfin_lasso.get_valid_data(X, y)
redfin_lasso.fit(X_train, y_train)
print("Validation Metrics")
redfin_lasso.return_metrics(X_valid, y_valid)
print("Test Metrics")
redfin_lasso.return_metrics(X_test, y_test)
print("Bootstrap Metrics")
X_test = np.array(X_test)
y_test = np.array(y_test)
redfin_lasso.return_bootstrap_metrics(
X_test,
y_test,
metrics=["r2", "explained_variance"],
n_samples=30,
num_resamples=300,
)
ridge_reg = Ridge()
estimator_name = "ridge"
# Set the parameters by cross-validation
ridge_pipeline_hyperparms_grid = {
f"{estimator_name}__max_iter": [100, 200, 500],
f"{estimator_name}__alpha": [0.1, 1, 0.5],
"selectKBest__k": [1, 2, 3],
}
kfold = False
calibrate = False
Model¶redfin_ridge = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="Redfin_model_Ridge",
estimator_name=estimator_name,
model_type="regression",
calibrate=calibrate,
estimator=ridge_reg,
kfold=kfold,
selectKBest=True,
stratify_y=False,
grid=ridge_pipeline_hyperparms_grid,
randomized_grid=False,
scoring=["r2"],
n_splits=5,
random_state=3,
)
redfin_ridge.grid_search_param_tuning(X, y)
### If KFold then the whole dataset is fed to the
### return metrics function
redfin_ridge.fit(X, y)
redfin_ridge.return_metrics(X, y)
redfin_ridge.predict(X_test)
sgd_reg = SGDRegressor(random_state=3)
estimator_name = "sgdregressor"
# Set the parameters by cross-validation
kfold = False
calibrate = False
sgd_pipeline_hyperparms_grid = [
{
"sgdregressor__loss": [
"squared_error",
"huber",
"epsilon_insensitive",
"squared_epsilon_insensitive",
],
"sgdregressor__penalty": [None, "l2", "l1", "elasticnet"][:1],
"sgdregressor__alpha": [0.0001, 0.001, 0.01, 0.1][:1],
"sgdregressor__l1_ratio": [
0.15,
0.25,
0.5,
0.75,
][
:1
], # Only used if penalty is 'elasticnet'
"sgdregressor__fit_intercept": [True, False][:1],
"sgdregressor__max_iter": [1000, 2000, 3000][:1],
"sgdregressor__tol": [1e-3, 1e-4][:1],
"sgdregressor__epsilon": [
0.1,
0.2,
], # Only used for 'huber' and 'epsilon_insensitive'
"sgdregressor__learning_rate": [
"constant",
"optimal",
"invscaling",
"adaptive",
][:1],
"sgdregressor__eta0": [
0.01,
0.1,
][:1],
"sgdregressor__power_t": [
0.25,
0.5,
][:1],
"sgdregressor__early_stopping": [True, False][:1],
"sgdregressor__validation_fraction": [
0.1,
0.2,
][:1],
"sgdregressor__n_iter_no_change": [
5,
10,
][:1],
"sgdregressor__warm_start": [True, False][:1],
"sgdregressor__average": [
False,
True,
10,
][:1],
}
]
Initialize and Configure the SGD Model
redfin_sgd = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="Redfin_model_SGD",
estimator_name=estimator_name,
model_type="regression",
calibrate=calibrate,
estimator=sgd_reg,
kfold=kfold,
stratify_y=False,
grid=sgd_pipeline_hyperparms_grid,
randomized_grid=False,
# n_iter=3,
scoring=["r2"],
# n_splits=2,
random_state=3,
)
redfin_sgd.grid_search_param_tuning(X, y)
X_train, y_train = redfin_sgd.get_train_data(X, y)
X_test, y_test = redfin_sgd.get_test_data(X, y)
X_valid, y_valid = redfin_sgd.get_valid_data(X, y)
redfin_sgd.fit(X_train, y_train)
print("Validation Metrics")
redfin_sgd.return_metrics(X_valid, y_valid)
print("Test Metrics")
redfin_sgd.return_metrics(X_test, y_test)
redfin_sgd.predict(X_test)
xg_boost = XGBRegressor(random_state=3)
estimator_name = "xgb"
# Set the parameters by cross-validation
kfold = False
calibrate = False
# Define the hyperparameters for XGBoost
xgb_learning_rates = [0.1, 0.01, 0.05][:1] # Learning rate or eta
xgb_n_estimators = [100, 200, 300][
:1
] # Number of trees. Equivalent to n_estimators in GB
xgb_max_depths = [3, 5, 7][:1] # Maximum depth of the trees
xgb_subsamples = [0.8, 1.0][:1] # Subsample ratio of the training instances
xgb_colsample_bytree = [0.8, 1.0][:1]
xgb_eval_metric = ["logloss"]
xgb_early_stopping_rounds = [10]
# xgb_tree_method = ["gpu_hist"]
# early_stopping_mode = ['min']
# early_stopping_patience = [5]
xgb_verbose = [False]
# Subsample ratio of columns when constructing each tree
# Combining the hyperparameters in a dictionary
xgb_pipeline_hyperparms_grid = [
{
"xgb__learning_rate": xgb_learning_rates,
"xgb__n_estimators": xgb_n_estimators,
"xgb__max_depth": xgb_max_depths,
"xgb__subsample": xgb_subsamples,
"xgb__colsample_bytree": xgb_colsample_bytree,
"xgb__eval_metric": xgb_eval_metric,
"xgb__early_stopping_rounds": xgb_early_stopping_rounds,
# 'xgb__early_stopping_patience': early_stopping_patience,
# "xgb_tree_method": xgb_tree_method,
"xgb__verbose": xgb_verbose,
}
]
Model¶redfin_xgb = Model(
pipeline_steps=[
("Preprocessor", SimpleImputer()),
],
name="Redfin_model_XGB",
estimator_name=estimator_name,
model_type="regression",
calibrate=calibrate,
estimator=xg_boost,
kfold=kfold,
stratify_y=False,
grid=xgb_pipeline_hyperparms_grid,
randomized_grid=False,
# n_iter=3,
scoring=["r2"],
# n_splits=2,
random_state=3,
xgboost_early=True,
)
eval_set = [X, y]
redfin_xgb.grid_search_param_tuning(X, y)
X_train, X_valid, X_test, y_train, y_valid, y_test = redfin_xgb.train_val_test_split(
X,
y,
stratify_y=False,
stratify_cols=None,
train_size=0.6,
validation_size=0.2,
test_size=0.2,
calibrate=redfin_xgb.calibrate,
random_state=redfin_xgb.random_state,
)
redfin_xgb.fit(X_train, y_train, validation_data=(X_valid, y_valid))
redfin_xgb.return_metrics(X_test, y_test)
Redfin. (n.d.). Redfin: Real Estate, Homes for Sale, MLS Listings, Agents. Retrieved from https://www.redfin.com