Custom model Evaluation Metric

In building machine learning models, There comes a time you might want to evaluate your model to know it's a performance using a metric you created. while scikit-learn with all it's beauty provides the evaluation metrics we need, there comes a time we might need to create our own metrics. scikit-learn makes this easy by providing a function make_scorer.
to create a custom metric, all you need to do is create a function containing the metric definition and convert it to a scorer function using scikit-learn make_scorer. we will use the make_regression dataset to show an example of creating a custom metric below.

let's begin by importing the neccessary libraries

# Load libraries
from sklearn.metrics import make_scorer, r2_score
from sklearn.model_selection import train_test_split
from sklearn.linear_model import Ridge
from sklearn.datasets import make_regression

Next we generate features and the target matrix.

features, target = make_regression(n_samples = 100,n_features = 4,
random_state = 1)

random_state parameter ensures we get the same distribution when we use the same random_state value.

let's continue by creating the training and testing set using train_test_split

X_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.10, random_state=1)

then create our custom metric function.

def custom_metric(X_test, target_predicted):
    # Calculate r-squared score
    r_squared = r2_score(X_test, target_predicted)
    # Return r-squared score
    return r_squared

next convert the custom metric into a scorer metric

# Make scorer and define that higher scores are better
score = make_scorer(custom_metric, greater_is_better=True)

greater_is_better signifies that higher score is better

Create the regression and train.

classifier = Ridge()
# Train ridge regression model
model = classifier.fit(X_train, y_train)

Finally apply the custom metric to check model performance.

score(model, X_test, y_test)