Eli5: Explain Image Classifier Predictions Using Grad-CAM (Keras)¶

Eli5 is one of the most commonly used libraries to interpret the predictions of Machine learning models. It let us interpret predictions of models created using scikit-learn, XGBoost, lightGBM, CatBoost, lightning, sklearn-crfsuite, and keras. We have covered a detailed tutorial explaining how we can use Eli5 for scikit-learn models (below link).

• How to Use eli5 to Understand sklearn Model Predictions?

As a part of this tutorial, we'll concentrate on the keras model. Currently Eli5 only supports image classifiers created using keras. It explains predictions of image classifier using Grad-CAM (Gradient-weighted Class Activation Mapping) algorithm. Grad-CAM algorithm creates a heatmap of shape same as the original image using gradients and output last convolution layer that we can visualize to see which parts of the image are contributing to the prediction. Eli5 lets us use Grad-CAM with only one function call. If the reader wants to know how Grad-CAM works internally then we recommend another tutorial where we have explained a step-by-step guide to Grad-CAM using PyTorch.

• PyTorch: Grad-CAM

In this tutorial, we have trained a simple convolutional neural network on the Fashion MNIST dataset. Then, we have explained the predictions of the network using Eli5's Grad-CAM implementation.

Below, we have highlighted important sections of tutorial to give an overview of the material covered.

Important Sections Of Tutorial¶

1. Load Dataset
2. Define And Train CNN
3. Evaluate Network Performance
4. Explain Prediction Using Eli5 Grad-CAM Implementation

Below, we have imported the necessary libraries and printed the versions that we have used in our tutorial.

Please make a NOTE that we have disabled tensorflow eager execution for Eli5 Grad-CAM implementation to work. It won't work with eager mode enabled.

1. Load Dataset ¶

Below, we have loaded the Fashion MNIST dataset available from keras. The dataset has grayscale images of shape (28,28) pixels for 10 different fashion items. The dataset is already divided into the train (60k images) and test (10k images) sets. Below we have included a table that has a mapping from target class index to target class name.

2. Define And Train CNN ¶

In this section, we have designed a small convolutional neural network to classify images of fashion MNIST dataset. The network has 3 convolution layers and one dense layer. The three convolution layers have 32, 16, and 8 output channels respectively. All of them apply kernel of size (3,3) on input data. All convolution layers apply relu activation to the output. The output of the third convolution layer is flattened and fed to a dense layer that has 10 output units (same as the target classes). The last dense layer has softmax activation which will transform the output of dense layers to probabilities.

After defining the network, we have compiled it to use Adam optimizer, cross entropy loss, and accuracy metric.

At last, we have trained the network for 8 epochs by giving train and validation data. We can notice from the accuracy getting printed after each epoch that our model is doing a good job at classifying images.

3. Evaluate Network Performance ¶

In this section, we have evaluated the performance of our network by calculating accuracy, confusion matrix and classification report metrics on test predictions. We can notice from the classification report and confusion matrix that our model is doing a good job for all categories except Shirt for which the accuracy is quite low compared to other categories.

We have calculated all metrics using various functions available from scikit-learn. Please feel free to check the below link if you want to know about various ML metrics available from sklearn in detail.

• Scikit-Learn - Model Evaluation & Scoring Metrics

4. Explain Prediction Using Eli5 Grad-CAM Implementation ¶

In this section, we have explained how we can use Grad-CAM algorithm available from Eli5 to explain predictions. The Grad-CAM algorithm returns a heatmap that can be overlayed over our original image to show which parts of the image are contributing to the prediction.

In order to use Grad-CAM algorithm available from Eli5 library, we need to call explain_prediction() function available from keras sub-module of Eli5 library. It'll return an instance of Explanation class which we can use to generate an image (using format_as_image() function) that has heatmap generated by Grad-CAM overlayed over original image. We can then visualize this image returned by format_as_image() function to see results.

Below, we have randomly selected a sample from data and made predictions on it. After making predictions, we have printed actual and predicted labels of the selected image. Then, we have called explain_prediction() function by giving it our model and selected sample. The method internally runs Grad-CAM algorithm and returns an Explanation object. The Explanation object has the original image and heatmap contained in it. Then, we have called format_as_image() function of Eli5 by giving it explanation object to generate final image. The final image has Grad-CAM heatmap overlayed on our original image which we have visualized.

In the below cell, we have called format_as_image() function again to generate images with a different colormap.

In the below cell, we have called format_as_image() function with a resampling filter to check whether it helps improve results.

By default, explain_prediction() function calculates heatmap using last convolution layer. But we can give convolution layer name as input asking it to generate heatmap with respect to some other convolution layer when executing Grad-CAM.

In the below cell, we have generated and visualized an image that was generated with respect to the second last convolution layer. We have given name of convolution layer to layer parameter of explain_prediction() function.

In the cell after the below cell, we have generated and visualized an image that was generated with respect to the first convolution layer.

Apart from explain_prediction(), eli5 provides another method named explain_prediction_keras() which works exactly like it. Below we have explained one example showing its usage.

Eli5 let us separately execute Grad-CAM algorithm as well and generate a heatmap by ourselves. It provides two functions for that purpose.

• gradcam_backend(model, sample,targets=None, activation_layer=None) - This function takes as input model, selected sample, target class index, and keras layer with respect to which execute Grad-CAM. It returns weights of the selected layer, activated output of the layer, gradients, predicted index, and predicted value as output.
• gradcam(weights, activations) - This function takes weights and activated values returned by gradcam_backend() function and executes Grad-CAM algorithm to generate heatmap.

Below, we have explained with a simple example how we can use them. After generating a heatmap, we have visualized it next to the original image for comparison purposes.

This ends our small tutorial explaining how we can use Grad-CAM implementation for keras networks available through Eli5. Please feel free to let us know your views in the comments section below.

References¶

• How to Use eli5 to Understand sklearn Model Predictions?
• PyTorch: Grad-CAM