Share @ LinkedIn Facebook  yellobrick, ml-metrics, visualization
Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Table of Contents

Yellowbrick

Python has many libraries that let us build machine learning models easily with a few lines of code. A library like scikit-learn has earned a reputation of the go-to library for ML models by the majority of data scientists and machine learning practitioners. Scikit-learn provides a very easy-to-use interface which lets us build an ML model using python with few lines of codes. Apart from that scikit-learn even provides functionalities related to feature selection, feature extraction, dimensionality reduction, grid searching hyper-parameters, etc. Though scikit-learn provides extensive models and metrics to evaluate those models, it does not provide functionalities to visualize that model evaluation metrics. Yellowbrick is a python library that provides various modules to visualize model evaluation metrics. Yellowbrick has different modules for tasks like feature visualizations, classification task metrics visualizations, regression task metrics visualizations, clustering task metrics visualizations, model selection visualizations, text data related visualizations, etc. We'll be explaining how to use yellowbrick API as a part of this tutorial with primarily concentrating on visualizing classification and regression task metrics.

  • Classification Metrics Visualizations
    • Confusion Matrix
    • Classification Report
    • ROC AUC Curve
    • Precision Recall Curve
    • Discrimination Threshold
    • Class Prediction Error
  • Regression Metrics Visualizations
    • Residual Plot
    • Prediction Error Plot
    • Alpha Selection
    • Cook's Distance

We'll start by importing the necessary libraries.

In [1]:
import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import sklearn
import yellowbrick

pd.set_option("display.max_columns", 35)

import warnings
warnings.filterwarnings("ignore")

Classification Metrics Visualizations

In this section, we'll be exploring classification metrics visualizations available with yellowbrick. We'll be using different datasets along with different sklearn estimators for this. We'll then print classification metrics visualizations explaining the performance of models on that dataset.

Load Datasets

We'll be using below mentioned three datasets for our purpose of explaining classification metrics visualizations which are easily available from sklearn:

  • Wine Dataset - It has information about ingredients used in three different types of wine.
  • Breast Cancer Dataset - It has information about tumor of breast cancer.
  • Digits Dataset - It has images of size 8x8 of digits 0-9.

Below we have loaded all datasets one by one and printed their descriptions explaining features of data. We have also kept datasets into pandas dataframe and displayed the first few data samples.

In [2]:
from sklearn.datasets import load_breast_cancer, load_wine, load_digits

wine = load_wine()

for line in wine.DESCR.split("\n")[5:29]:
    print(line)

wine_df = pd.DataFrame(data=wine.data, columns=wine.feature_names)
wine_df["WineType"] = wine.target

wine_df.head()
**Data Set Characteristics:**

    :Number of Instances: 178 (50 in each of three classes)
    :Number of Attributes: 13 numeric, predictive attributes and the class
    :Attribute Information:
 		- Alcohol
 		- Malic acid
 		- Ash
		- Alcalinity of ash
 		- Magnesium
		- Total phenols
 		- Flavanoids
 		- Nonflavanoid phenols
 		- Proanthocyanins
		- Color intensity
 		- Hue
 		- OD280/OD315 of diluted wines
 		- Proline

    - class:
            - class_0
            - class_1
            - class_2

Out[2]:
alcohol malic_acid ash alcalinity_of_ash magnesium total_phenols flavanoids nonflavanoid_phenols proanthocyanins color_intensity hue od280/od315_of_diluted_wines proline WineType
0 14.23 1.71 2.43 15.6 127.0 2.80 3.06 0.28 2.29 5.64 1.04 3.92 1065.0 0
1 13.20 1.78 2.14 11.2 100.0 2.65 2.76 0.26 1.28 4.38 1.05 3.40 1050.0 0
2 13.16 2.36 2.67 18.6 101.0 2.80 3.24 0.30 2.81 5.68 1.03 3.17 1185.0 0
3 14.37 1.95 2.50 16.8 113.0 3.85 3.49 0.24 2.18 7.80 0.86 3.45 1480.0 0
4 13.24 2.59 2.87 21.0 118.0 2.80 2.69 0.39 1.82 4.32 1.04 2.93 735.0 0
In [3]:
breast_cancer = load_breast_cancer()

for line in breast_cancer.DESCR.split("\n")[5:31]:
    print(line)

breast_cancer_df = pd.DataFrame(data=breast_cancer.data, columns=breast_cancer.feature_names)
breast_cancer_df["CancerType"] = breast_cancer.target

breast_cancer_df.head()
**Data Set Characteristics:**

    :Number of Instances: 569

    :Number of Attributes: 30 numeric, predictive attributes and the class

    :Attribute Information:
        - radius (mean of distances from center to points on the perimeter)
        - texture (standard deviation of gray-scale values)
        - perimeter
        - area
        - smoothness (local variation in radius lengths)
        - compactness (perimeter^2 / area - 1.0)
        - concavity (severity of concave portions of the contour)
        - concave points (number of concave portions of the contour)
        - symmetry
        - fractal dimension ("coastline approximation" - 1)

        The mean, standard error, and "worst" or largest (mean of the three
        largest values) of these features were computed for each image,
        resulting in 30 features.  For instance, field 3 is Mean Radius, field
        13 is Radius SE, field 23 is Worst Radius.

        - class:
                - WDBC-Malignant
                - WDBC-Benign
Out[3]:
mean radius mean texture mean perimeter mean area mean smoothness mean compactness mean concavity mean concave points mean symmetry mean fractal dimension radius error texture error perimeter error area error smoothness error compactness error concavity error concave points error symmetry error fractal dimension error worst radius worst texture worst perimeter worst area worst smoothness worst compactness worst concavity worst concave points worst symmetry worst fractal dimension CancerType
0 17.99 10.38 122.80 1001.0 0.11840 0.27760 0.3001 0.14710 0.2419 0.07871 1.0950 0.9053 8.589 153.40 0.006399 0.04904 0.05373 0.01587 0.03003 0.006193 25.38 17.33 184.60 2019.0 0.1622 0.6656 0.7119 0.2654 0.4601 0.11890 0
1 20.57 17.77 132.90 1326.0 0.08474 0.07864 0.0869 0.07017 0.1812 0.05667 0.5435 0.7339 3.398 74.08 0.005225 0.01308 0.01860 0.01340 0.01389 0.003532 24.99 23.41 158.80 1956.0 0.1238 0.1866 0.2416 0.1860 0.2750 0.08902 0
2 19.69 21.25 130.00 1203.0 0.10960 0.15990 0.1974 0.12790 0.2069 0.05999 0.7456 0.7869 4.585 94.03 0.006150 0.04006 0.03832 0.02058 0.02250 0.004571 23.57 25.53 152.50 1709.0 0.1444 0.4245 0.4504 0.2430 0.3613 0.08758 0
3 11.42 20.38 77.58 386.1 0.14250 0.28390 0.2414 0.10520 0.2597 0.09744 0.4956 1.1560 3.445 27.23 0.009110 0.07458 0.05661 0.01867 0.05963 0.009208 14.91 26.50 98.87 567.7 0.2098 0.8663 0.6869 0.2575 0.6638 0.17300 0
4 20.29 14.34 135.10 1297.0 0.10030 0.13280 0.1980 0.10430 0.1809 0.05883 0.7572 0.7813 5.438 94.44 0.011490 0.02461 0.05688 0.01885 0.01756 0.005115 22.54 16.67 152.20 1575.0 0.1374 0.2050 0.4000 0.1625 0.2364 0.07678 0
In [4]:
digits = load_digits()

for line in digits.DESCR.split("\n")[5:29]:
    print(line)
**Data Set Characteristics:**

    :Number of Instances: 5620
    :Number of Attributes: 64
    :Attribute Information: 8x8 image of integer pixels in the range 0..16.
    :Missing Attribute Values: None
    :Creator: E. Alpaydin (alpaydin '@' boun.edu.tr)
    :Date: July; 1998

This is a copy of the test set of the UCI ML hand-written digits datasets
https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits

The data set contains images of hand-written digits: 10 classes where
each class refers to a digit.

Preprocessing programs made available by NIST were used to extract
normalized bitmaps of handwritten digits from a preprinted form. From a
total of 43 people, 30 contributed to the training set and different 13
to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of
4x4 and the number of on pixels are counted in each block. This generates
an input matrix of 8x8 where each element is an integer in the range
0..16. This reduces dimensionality and gives invariance to small
distortions.

Split Datasets into Train/Test Sets

We have now split all three datasets mentioned above into train (80%) and test (20%) sets using the train_test_split() method of sklearn. We'll be using these train/test sets for training models and evaluating performance.

In [5]:
from sklearn.model_selection import train_test_split

X_wine, Y_wine = wine.data, wine.target

print("Wine Dataset Size : ", X_wine.shape, Y_wine.shape)

X_train_wine, X_test_wine, Y_train_wine, Y_test_wine = train_test_split(X_wine, Y_wine, train_size=0.80, stratify=Y_wine, random_state=123)

print("Wine Train/Test Sizes : ", X_train_wine.shape, X_test_wine.shape, Y_train_wine.shape, Y_test_wine.shape)

print()

X_breast_can, Y_breast_can = breast_cancer.data, breast_cancer.target

print("Breast Cancer Dataset Size : ", X_breast_can.shape, Y_breast_can.shape)

X_train_breast_can, X_test_breast_can, Y_train_breast_can, Y_test_breast_can = train_test_split(X_breast_can, Y_breast_can, train_size=0.80, stratify=Y_breast_can, random_state=123)

print("Breast Cancer Train/Test Sizes : ", X_train_breast_can.shape, X_test_breast_can.shape, Y_train_breast_can.shape, Y_test_breast_can.shape)

print()

X_digits, Y_digits = digits.data, digits.target

print("Digits Dataset Size : ", X_breast_can.shape, Y_breast_can.shape)

X_train_digits, X_test_digits, Y_train_digits, Y_test_digits = train_test_split(X_digits, Y_digits, train_size=0.80, stratify=Y_digits, random_state=123)

print("Digits Train/Test Sizes : ", X_train_digits.shape, X_test_digits.shape, Y_train_digits.shape, Y_test_digits.shape)
Wine Dataset Size :  (178, 13) (178,)
Wine Train/Test Sizes :  (142, 13) (36, 13) (142,) (36,)

Breast Cancer Dataset Size :  (569, 30) (569,)
Breast Cancer Train/Test Sizes :  (455, 30) (114, 30) (455,) (114,)

Digits Dataset Size :  (569, 30) (569,)
Digits Train/Test Sizes :  (1437, 64) (360, 64) (1437,) (360,)

Confusion Matrix

The first chart that we'll introduce is a confusion matrix plot. The classifier module of yellowbrick has a class named ConfusionMatrix which lets us create a confusion matrix chart. We'll first need to create an object of this class passing it machine learning model. We can then call the fit() and score() method on the object of class ConfusionMatrix which will train model passed to it on train data and evaluate it on test data. We can then simply call the show() method on this object and it'll create a confusion matrix of test data. We have generated a confusion matrix of digits test data and used a random forest sklearn estimator.

Please make a note that the show() method will always show charts based on the data set on which the score() method was called. Below we have called the score() method with the test dataset. If we want to generate a confusion matrix for train data then we need to call the score() method with train data.

The show method has a parameter named output_path where we can path and image name if we want to store image to disk.

In [ ]:
from yellowbrick.classifier import ConfusionMatrix
from sklearn.ensemble import RandomForestClassifier

visualizer = ConfusionMatrix(RandomForestClassifier(random_state=1), classes=digits.target_names)

visualizer.fit(X_train_digits, Y_train_digits)

visualizer.score(X_test_digits, Y_test_digits)

visualizer.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created another example explaining the usage of the confusion matrix. We have also modified a few important parameters of the ConfusionMatrix.

  • percent - It accepts boolean specifying whether to show count or percent in the matrix.
  • cmap - It accepts a string of matplotlib colormap object specifying matplotlib color.
  • fontsize - It accepts integer specifying the size of the font in the chart.
  • ax - It accepts matplotlib axes on which to draw the chart.
In [ ]:
fig = plt.figure(figsize=(7,7))
ax = fig.add_subplot(111)

visualizer = ConfusionMatrix(RandomForestClassifier(random_state=123),
                             classes=digits.target_names,
                             percent=True,
                             cmap="Blues",
                             fontsize=13,
                             ax=ax)

visualizer.fit(X_train_digits, Y_train_digits)

visualizer.score(X_test_digits, Y_test_digits)

visualizer.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

The yellowbrick also provides another way of creating a chart by using methods if we don't want to create a class object. Below we have created a confusion matrix by using the confusion_matrix() method available.

Please make a note that we have passed the sklearn estimator, train data, test data, and few other parameters to the method. The passing of test data is optional. If we don't pass test data then it'll create a matrix based on train data.

In [ ]:
from yellowbrick.classifier.confusion_matrix import confusion_matrix

fig = plt.figure(figsize=(7,7))
ax = fig.add_subplot(111)

confusion_matrix(RandomForestClassifier(random_state=123),
                                          X_train_digits, Y_train_digits,
                                          X_test_digits, Y_test_digits,
                                          percent=True,
                                          classes=digits.target_names,
                                          fontsize=13,
                                          ax=ax,
                                          cmap="Blues",
                                         );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Classification Report

The second chart type that we'll explain using yellowbrick is a classification report which shows precision, recall, support, and f1-score for each individual class of classification problem.

We'll be following the same process to create a visualization that we followed in the previous example. We'll first create an object of class ClassificationReport passing it sklearn estimator and list of class names. We'll then fit that object with train data and evaluate using test data. We'll then call the show() method to generate the figure. We have generated a classification report of digits test data and used a sklearn decision tree estimator for training data.

In [ ]:
from yellowbrick.classifier import ClassificationReport
from sklearn.tree import DecisionTreeClassifier

viz = ClassificationReport(DecisionTreeClassifier(random_state=123),
                           classes=wine.target_names,
                           support=True,
                           fig=plt.figure(figsize=(8,6)))

viz.fit(X_train_wine, Y_train_wine)

viz.score(X_test_wine, Y_test_wine)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have explained another way of creating a classification report chart using the classification_report() method of yellowbrick.

In [ ]:
from yellowbrick.classifier.classification_report import classification_report

classification_report(DecisionTreeClassifier(random_state=123),
                      X_train_wine, Y_train_wine,
                      X_test_wine, Y_test_wine,
                      classes=wine.target_names,
                      support="percent",
                      cmap="Reds",
                      font_size=16,
                      fig=plt.figure(figsize=(8,6))
                     );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

ROC AUC Curve

The third chart type that we'll explain is the ROC AUC curve chart. We have followed the same step of creating a chart as earlier examples. We have first created an object of class ROCAUC passing it sklearn decision tree estimator, fir object to train data, evaluated it on test data and plotted figure of test data by calling show() method.

Below are some important parameters of the ROCAUC class:

  • micro - It accepts boolean value specifying whether to include micro-averages ROC curve or not. It’s computed from the sum of all true positives and all false positives across all classes of data. The default is True.
  • macro - It accepts boolean value specifying whether to include macro-averages ROC curve or not. It’s computed by taking an average of all individual class ROC curves. The default is True.
  • per_class - It accepts boolean value specifying whether to plot ROC curve for an individual class or not. It’s True by default. If set to False then only micro and macro ROC curves are displayed.
In [ ]:
from yellowbrick.classifier import ROCAUC

viz = ROCAUC(DecisionTreeClassifier(random_state=123),
             classes=breast_cancer.target_names,
             fig=plt.figure(figsize=(7,5)))

viz.fit(X_train_breast_can, Y_train_breast_can)

viz.score(X_test_breast_can, Y_test_breast_can)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created the ROC AUC curve using the roc_auc() method of yellowbrick.

In [ ]:
from yellowbrick.classifier.rocauc import roc_auc, roc_curve

fig = plt.figure(figsize=(10,7))
ax1 = fig.add_subplot(111)

roc_auc(DecisionTreeClassifier(random_state=2),
        X_train_digits, Y_train_digits,
        X_test_digits, Y_test_digits,
        classes=digits.target_names,
        ax=ax1
        );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

In [ ]:
roc_auc(DecisionTreeClassifier(random_state=2),
        X_train_wine, Y_train_wine,
        X_test_wine, Y_test_wine,
        classes=wine.target_names,
        per_class=False,
        );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Precision Recall Curve

The fourth chart type that we'll explain is precision-recall curves. We have followed the same process as previous examples to create this chart. We have created an object of class PrecisionRecallCurve with a sklearn decision tree estimator, the trained object on breast cancer train data, and evaluated it on test data. We have then called the show() method to display the chart.

Below is a list of some important parameters of class PrecisionRecallCurve:

  • fill_area - It accepts boolean specifying whether to fill the area under the curve. The default is True.
  • ap_score - It accepts boolean specifying whether to annotate the graph with an average precision score or not. The default is True.
  • micro - It accepts boolean value specifying whether to include micro-averages ROC curve or not. It’s computed from the sum of all true positives and all false positives across all classes of data. The default is True.
  • per_class - It accepts boolean value specifying whether to plot ROC curve for an individual class or not. It’s True by default. If set to False then only micro and macro ROC curves are displayed. The default is False.
  • iso_f1_curves - It accepts boolean value specifying whether to include ISO F1-curves on a chart or not. The default is False.
  • iso_f1_values - It accepts tuple specifying values of f1 score for which to draw ISO F1-curves. The default is (0.2, 0.4, 0.6, 0.8).
In [ ]:
from yellowbrick.classifier import PrecisionRecallCurve

viz = PrecisionRecallCurve(DecisionTreeClassifier(random_state=123),
             classes=breast_cancer.target_names,
             ap_score=True,
             iso_f1_curves=True,
             fig=plt.figure(figsize=(7,5)))

viz.fit(X_train_breast_can, Y_train_breast_can)

viz.score(X_test_breast_can, Y_test_breast_can)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created a precision-recall curve using the precision_recall_curve() method of yellowbrick on digits test data.

In [ ]:
from yellowbrick.classifier.prcurve import precision_recall_curve

precision_recall_curve(DecisionTreeClassifier(random_state=123),
                       X_train_digits, Y_train_digits,
                       X_test_digits, Y_test_digits,
                       classes=digits.target_names,
                       per_class=True,
                       fill_area=False,
                       iso_f1_curves=True,
                       fig=plt.figure(figsize=(12,8))
                       );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Discrimination Threshold

The fifth chart type that we'll introduce is the discrimination threshold. It displays how precision, recall, f1 score, and queue rate change as we change the threshold at which we decide class prediction. The output of the machine learning model for classification problems is generally probability and we decide class based on some threshold on probability. The sklearn has put the threshold generally at 0.5 which means that if the probability is greater than 0.5 then we take the class as positive class else negative class.

Below we have created a discrimination threshold chart by creating an object of class DiscriminationThreshold passing it sklearn logistic regression estimator. We have then fitted object on breast cancer train data and evaluated it on test data before generating a chart.

In [ ]:
from sklearn.linear_model import LogisticRegression
from yellowbrick.classifier import DiscriminationThreshold


viz = DiscriminationThreshold(LogisticRegression(random_state=123),
             classes=breast_cancer.target_names,
             cv=0.2,
             fig=plt.figure(figsize=(9,6)))

viz.fit(X_train_breast_can, Y_train_breast_can)

viz.score(X_test_breast_can, Y_test_breast_can)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Class Prediction Error

The sixth and last chart type that we'll introduce for classification metrics visualizations is class prediction error. It’s a bar chart showing how many samples are correctly classified and how many wrongly along with a class in which they were wrongly classified.

We have created a chart exactly the same way as earlier. We have created an object of class ClassPredictionError passing it a sklearn decision tree classifier. We have then fitted object on wine train data and evaluated it on test data. We have then plotted the class prediction error chart of test data. We can see from the below chart that few class_1 confused with class_0 from the first bar. Few class_2 samples and few class_0 samples are confused with class_1 from the second bar. The third bar tells us that other classes are not confused with class_2 at least.

In [ ]:
from yellowbrick.classifier import ClassPredictionError

viz = ClassPredictionError(DecisionTreeClassifier(random_state=123),
                           classes=wine.target_names,
                           fig=plt.figure(figsize=(9,6)))

viz.fit(X_train_wine, Y_train_wine)

viz.score(X_test_wine, Y_test_wine)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Regression Metrics Visualizations

As a part of this section, we'll be explaining various regression metrics visualizations. We'll be loading different datasets and using different sklearn regression estimators for this.

Load Datasets

We'll be using below mentioned two datasets which are easily available from scikit-learn for the creation of various regression metrics visualizations:

  • Boston Housing Price - It has information about various house attributes for houses in Boston.
  • California Housing - It has information about various house attributes for houses in California.

Below we have loaded all datasets one by one and printed their descriptions explaining features of data. We have also kept datasets into pandas dataframe and displayed the first few data samples.

In [18]:
from sklearn.datasets import load_boston

boston = load_boston()

for line in boston.DESCR.split("\n")[5:26]:
    print(line)

boston_df = pd.DataFrame(data=boston.data, columns=boston.feature_names)
boston_df["Price($)"] = boston.target

boston_df.head()
**Data Set Characteristics:**

    :Number of Instances: 506

    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's
Out[18]:
CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX PTRATIO B LSTAT Price($)
0 0.00632 18.0 2.31 0.0 0.538 6.575 65.2 4.0900 1.0 296.0 15.3 396.90 4.98 24.0
1 0.02731 0.0 7.07 0.0 0.469 6.421 78.9 4.9671 2.0 242.0 17.8 396.90 9.14 21.6
2 0.02729 0.0 7.07 0.0 0.469 7.185 61.1 4.9671 2.0 242.0 17.8 392.83 4.03 34.7
3 0.03237 0.0 2.18 0.0 0.458 6.998 45.8 6.0622 3.0 222.0 18.7 394.63 2.94 33.4
4 0.06905 0.0 2.18 0.0 0.458 7.147 54.2 6.0622 3.0 222.0 18.7 396.90 5.33 36.2
In [19]:
from sklearn.datasets import fetch_california_housing

calif_housing = fetch_california_housing()

for line in calif_housing.DESCR.split("\n")[5:26]:
    print(line)

calif_housing_df = pd.DataFrame(data=calif_housing.data, columns=calif_housing.feature_names)
calif_housing_df["Price($)"] = calif_housing.target

calif_housing_df.head()
**Data Set Characteristics:**

    :Number of Instances: 20640

    :Number of Attributes: 8 numeric, predictive attributes and the target

    :Attribute Information:
        - MedInc        median income in block
        - HouseAge      median house age in block
        - AveRooms      average number of rooms
        - AveBedrms     average number of bedrooms
        - Population    block population
        - AveOccup      average house occupancy
        - Latitude      house block latitude
        - Longitude     house block longitude

    :Missing Attribute Values: None

This dataset was obtained from the StatLib repository.
http://lib.stat.cmu.edu/datasets/

Out[19]:
MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude Longitude Price($)
0 8.3252 41.0 6.984127 1.023810 322.0 2.555556 37.88 -122.23 4.526
1 8.3014 21.0 6.238137 0.971880 2401.0 2.109842 37.86 -122.22 3.585
2 7.2574 52.0 8.288136 1.073446 496.0 2.802260 37.85 -122.24 3.521
3 5.6431 52.0 5.817352 1.073059 558.0 2.547945 37.85 -122.25 3.413
4 3.8462 52.0 6.281853 1.081081 565.0 2.181467 37.85 -122.25 3.422

Split Datasets into Train/Test Sets

Below we have split both datasets loaded earlier into the train (80%) and test (20%) sets.

In [20]:
from sklearn.model_selection import train_test_split

X_boston, Y_boston = boston.data, boston.target

print("Boston Dataset Size : ", X_boston.shape, Y_boston.shape)

X_train_boston, X_test_boston, Y_train_boston, Y_test_boston = train_test_split(X_boston, Y_boston, train_size=0.80, random_state=123)

print("Boston Train/Test Sizes : ", X_train_boston.shape, X_test_boston.shape, Y_train_boston.shape, Y_test_boston.shape)

print()

X_calif_housing, Y_calif_housing = calif_housing.data, calif_housing.target

print("California Housing Dataset Size : ", X_calif_housing.shape, Y_calif_housing.shape)

X_train_calif_housing, X_test_calif_housing, Y_train_calif_housing, Y_test_calif_housing = train_test_split(X_calif_housing, Y_calif_housing, train_size=0.80, random_state=123)

print("California Housing Train/Test Sizes : ", X_train_calif_housing.shape, X_test_calif_housing.shape, Y_train_calif_housing.shape, Y_test_calif_housing.shape)
Boston Dataset Size :  (506, 13) (506,)
Boston Train/Test Sizes :  (404, 13) (102, 13) (404,) (102,)

California Housing Dataset Size :  (20640, 8) (20640,)
California Housing Train/Test Sizes :  (16512, 8) (4128, 8) (16512,) (4128,)

Residual Plot

The first chart type that we'll introduce for explaining regression metrics visualizations is the residual plot. The residual plots show a scatter plot between the predicted value on x-axis and residual on the y-axis. It points that if points are randomly distributed across the horizontal axis then it’s advisable to choose linear regression for it else a non-linear model will be an appropriate choice.

We have followed the same process as previous examples to create this chart. We have created an object of class ResidualsPlot, fitted it on Boston housing train data, and evaluated it on test data.

In [ ]:
from yellowbrick.regressor import ResidualsPlot
from sklearn.ensemble import RandomForestRegressor

viz = ResidualsPlot(RandomForestRegressor(random_state=123),
                    train_color="dodgerblue",
                    test_color="tomato",
                    fig=plt.figure(figsize=(9,6))
                    )

viz.fit(X_train_boston, Y_train_boston)
viz.score(X_test_boston, Y_test_boston)
viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created a residuals plot using the residuals_plot() method on California housing data. We have used a sklearn random forest regressor this time.

In [ ]:
from yellowbrick.regressor.residuals import residuals_plot

residuals_plot(RandomForestRegressor(random_state=123),
               X_train_calif_housing, Y_train_calif_housing,
               X_test_calif_housing, Y_test_calif_housing,
               train_color="tomato",
               test_color="lime",
               line_color="dodgerblue",
               fig=plt.figure(figsize=(9,6)));

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Prediction Error Plot

The second chart type that we'll introduce is the prediction error plot which is a scatter plot of actual target values on the x-axis and model predicted values on the y-axis.

We have created a chart by first creating an object of class PredictionError. We have then fitted Boston train data and evaluated test data. The chart also has boolean parameters named bestfit and identity which specifies whether to include the best fit line and identity line in the chart or not.

In [ ]:
from yellowbrick.regressor import PredictionError
from sklearn.ensemble import GradientBoostingRegressor

viz = PredictionError(GradientBoostingRegressor(random_state=123),
                      fig=plt.figure(figsize=(6,6))
                      )

viz.fit(X_train_boston, Y_train_boston)
viz.score(X_test_boston, Y_test_boston)
viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created a prediction error plot using the prediction_error() method.

In [ ]:
from yellowbrick.regressor.prediction_error import prediction_error

prediction_error(GradientBoostingRegressor(random_state=123),
                 X_train_calif_housing, Y_train_calif_housing,
                 X_test_calif_housing, Y_test_calif_housing,
                 fig=plt.figure(figsize=(7,7))
                 );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Alpha Selection

The third chart type that we'll explain for regression tasks is the alpha selection chart which demonstrates how different values of alpha influence model selection during regularization. It shows the impact of regularization on the model. This will work on RegressionCV models of sklearn which are RidgeCV and LassoCV. We can try different alpha values.

We have created a chart by using an object of class AlphaSelection using the same process as previous examples.

In [ ]:
from yellowbrick.regressor import AlphaSelection
from sklearn.linear_model import RidgeCV

viz = AlphaSelection(RidgeCV(),
                     fig=plt.figure(figsize=(7,7))
                    )

viz.fit(X_train_boston, Y_train_boston)
viz.score(X_test_boston, Y_test_boston)
viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created the alpha selection chart using the alphas() method of yellowbrick.

In [ ]:
from yellowbrick.regressor.alphas import alphas
from sklearn.linear_model import LassoCV

alphas(LassoCV(),
       X_calif_housing, Y_calif_housing,
       fig=plt.figure(figsize=(7,7))
       );

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Cook's Distance

The fourth and last chart type that we'll introduce is the cook's distance chart which displays the importance of individual features on the model. The removal of high influential samples from data might change model coefficients hence performance. Its generally used to detect outliers that can influence the model in a particular direction with the presence of just a few samples.

Below we have created a cook's distance chart using CooksDistance object. We have created it on the Boston housing dataset.

In [ ]:
from yellowbrick.regressor import CooksDistance

viz = CooksDistance(fig=plt.figure(figsize=(9,7)))

viz.fit(X_boston, Y_boston)

viz.show();

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python

Below we have created a cook's distance chart using the cooks_distance() method of yellowbrick.

In [ ]:
from yellowbrick.regressor import cooks_distance

cooks_distance(
    X_calif_housing, Y_calif_housing,
    draw_threshold=True,
    linefmt="C0-", markerfmt=",",
    fig = plt.figure(figsize=(12,7))
);

Yellowbrick - Visualize Sklearn's Classification & Regression Metrics in Python



Sunny Solanki  Sunny Solanki