LIME: Explain Keras Image Classification Network (CNN) Predictions¶

LIME (Local Interpretable Model-Agnostic Explanations) is one of the most commonly used algorithms to explain the predictions of black-box models. We can generate feature importances when we are using ML models like linear regression, decision trees, random forests, gradient boosting trees, etc. But when we are using deep neural networks with many layers, it's hard to generate feature importances specifying how much each feature contributed to particular predictions. In those situations, LIME can help us. It internally generates a few fake samples from the input sample, trains a simple ML model (decision tree, linear regression, etc.) that can mimic the prediction of our black-box model (deep neural network), and use that simple model's prediction to explain the prediction of our black-box model. This helps us generate feature importances for our black-box model. We have a separate tutorial where we have covered how LIME works in detail. Please feel free to check it from the below link.

• How to Use LIME to Understand sklearn Models Predictions?

As a part of this tutorial, we'll be primarily concentrating on image classification tasks. We have created a keras convolution neural network and trained it on the Fashion MNIST dataset. Later on, we have used LIME to explain the predictions made by the model. We have plotted visualizations showing which parts of the images are contributing to prediction.

Below, we have listed important sections of our tutorial to give an overview of the material covered.

Important Sections Of Tutorial¶

1. Load Data
2. Create And Train Model
3. Evaluate Network Performance
4. Explain Predictions Using Lime Image Explainer
   • 4.1 Explain True Prediction
     • Pixels Contributing Positively to Prediction
     • Pixels Contributing Negatively to Prediction
   • 4.2 Explain True Predictions With Segmentation Method
   • 4.3 Explain Wrong Prediction

Below, we have imported the necessary libraries and printed the versions that we have used in our tutorial.

1. Load Data ¶

In this section, we have loaded the Fashion MNIST dataset available from keras. The dataset has grayscale images of shape (28,28) pixels of 10 different fashion items. The dataset is already divided into the train (60k images) and test (10k images) sets. Below, we have included a table that has a mapping from index to item names.

2. Create And Train Model ¶

In this section, we have created a Convolution neural network for the image classification tasks and trained it. The network consists of 3 convolution layers and 1 dense layer. The three convolution layers have 48, 32, and 16 filters respectively with a kernel size of (3,3). All convolution layers have relu activation function. The output of the third convolution layer is flattened and given to the dense layer which has 10 units as input. The dense layer has softmax activation function.

After creating and initializing the network, we have compiled it to use Adam optimizer, cross entropy loss, and accuracy metric.

At last, we have trained the network for 10 epochs with a batch size of 256. We can notice from loss and accuracy getting printed after each epoch that the model is doing a good job at the image classification task.

3. Evaluate Network Performance ¶

In this section, we have evaluated the performance of the network by calculating accuracy, confusion matrix and classification report metrics on test predictions. We can notice from test accuracy that the model is doing a good job overall. When we look closely at the classification report and confusion matrix, we can notice that model is confused in categories T-shirt/top, Pullover, and Shirt. They have a little less accuracy compared to other categories.

We have used various functions available from scikit-learn to calculate various ML metrics. Please feel free to check the below link if you want to learn about them as the tutorial covers the majority of them.

• Scikit-Learn - Model Evaluation & Scoring Metrics

4. Explanation Using Lime Image Explainer ¶

In this section, we have explained predictions made by our model using an image explainer available from lime python library. In order to explain prediction using lime, we need to create an instance of LimeImageExplainer. Then, we can call explain_instance() method on it to create an instance of Explanation. The Explanation instance has details about pixels that contributed to prediction. We can call get_image_and_mask() method on Explanation instance to generate an image and a mask that can be plotted to see which pixels contributed to prediction.

Below, we have first created an instance of LimeImageExplainer using LimeImageExplainer() constructor available from lime_image sub-module of lime. We have not covered in detail various parameters of LimeImageExplainer() constructor here. Please feel free to check the below link if you want to tweak default settings.

• LimeImageExplainer

Create Prediction Function¶

In this section, we have created a simple function that takes images as input and makes predictions. This function will be required by explain_instance() method later. It simply returns 10 probabilities for each sample.

Please make a NOTE that our function takes as an input color image and works on it by converting it to a grayscale image. The reason behind this is that explain_instance() function of LimeImageExplainer can take the grayscale image as input but it converts the grayscale image to color before passing it to the prediction function.

4.1 Explain True Prediction ¶

In this section, we have explained the correct prediction made by our model. We have created separate visualizations showing pixels that contribute positively and negatively to prediction.

Below, we have first randomly selected a sample from test data. Then, we have printed the actual label of data and the predicted label. Then, we have called explain_instance() function with the test sample and prediction function. It generated an instance of Explanation which we'll use to generate an image and mask for visualization purposes next.

Pixels Contributing Positively to Prediction¶

In this section, we have generated an image and mask that has pixels contributing positively to the prediction highlighted. We have given actual label as input to the get_image_and_mask() function of Explanation instance.

Then, in the next cell, we have displayed the actual image, the image returned by get_image_and_mask() function, mask, and combination of image and mask. This will help us compare and see which pixels contributed to the prediction. We can notice that the top and bottom pixels of the t-shirt are contributing to prediction positively.

Pixels Contributing Negatively to Prediction¶

In this section, we have generated visualization showing pixels that contributes negatively to the prediction category. We have set negative_only parameter of method get_image_and_mask() function to True and positive_only parameter to False. We can notice from the visualization that some middle pixels are contributing negatively to prediction which means that they could be positively contributing to some other category.

4.2 Explain True Predictions With Segmentation Method ¶

In this section, we have again explained correct prediction but this time, we have provided different segmentation methods to explain_instance() function to see whether it makes any difference. We have used felzenszwalb segmentation method available from scikit-image library. We have generated a new explanation instance for a random test sample using felzenszwalb segmentation method.

Pixels Contributing Positively to Prediction¶

In this section, we have created a visualization showing pixels contributing positively to the prediction category using an Explanation object created using felzenszwalb segmentation method.

Pixels Contributing Negatively to Prediction¶

In this section, we have created a visualization showing pixels contributing negatively to the prediction category using an Explanation object created using felzenszwalb segmentation method.

4.3 Explain Wrong Prediction ¶

In this section, we have explained a wrong prediction made by our model. This will help us better understand which pixels are contributing to the prediction of the wrong category.

Below, we have first retrieved indexes of samples that are predicted wrong by our model. Then, we have randomly selected a sample that is predicted wrong by our model. The actual category of the selected sample is Pullover but our model predicts T-shirt/top. We have created an explanation instance as usual.

Pixels Contributing Positively to Prediction¶

In this section, we have created a visualization showing pixels that contribute positively to the actual category of the sample. We can notice from the results that it's missing important middle pixels that could have contributed to predicting the category Pullover.

Pixels Contributing Negatively to Prediction¶

In this section, we have created a visualization showing pixels that contribute negatively to predicting the actual category of the sample. We can notice that middle pixels which should have contributed positively to the prediction category are contributing negatively. This highlights that we need to work on our model and improve it further so that it can catch these kinds of patterns.

This ends our small tutorial explaining how we can interpret the predictions made by the keras image classification network using LIME. Please feel free to let us know your views in the comments section.

References¶

• How to Use LIME to Understand sklearn Models Predictions?
• Convolutional Neural Networks (CNN or ConvNet)
• Scikit-Learn - Model Evaluation & Scoring Metrics