Contrastive Explanations in Neural Networks

 Personnel: Mohit Prabhushankar, Jinsol Lee, Yutong Sun

Goal: To construct interventions that provide contextually relevant explanations

Challenges: Visual explanations are generally seen as direct answers to the question `Why P?’ posed to the user, where P is any given decision. Such an explanation does not require any other interaction from the user nor does it require the user to have prior knowledge about the data and classes. Because of this lack of meaningful interaction, such explanations are limited. Ideally, explanations are targeted towards a user. Such explanations derive context and relevance from the user. However, interaction requires interventions which are challenging.

Our work: Current `Why P?’ questions operate under broad contexts thereby providing answers that are irrelevant in some cases. We propose to constrain these Why questions based on some context Q so that our explanations answer contrastive questions of the form `Why P, rather than Q?' [1]. In this work, we formalize the structure of contrastive visual explanations for neural networks. We define contrast based on neural networks and propose a methodology to extract defined contrasts. We use loss gradients of parameters as contrast. We then use the extracted contrasts as a plug-in on top of existing `Why P?' techniques, specifically Grad-CAM and demonstrate their value in analyzing both networks and data in applications of large-scale recognition, fine-grained recognition, subsurface seismic analysis, and image quality assessment.

Extensions: Contrastive explanations [1] intervene based on a contrastive Q class. In [2], we place the contrastive explanations within the larger literature of correlations, and counterfactuals and show that observed paradigms of contrastive, correlations, and counterfactual explanations provide complete explanations. We also provide a taxonomy of evaluating explanations in [2]. In [3], we show that backpropagating the power set of all possible labels leads to visual causal features. The usage of gradients have also been explored in human saliency detection where contrast is interpreted as the expectancy-mismatch hypothesis [4]. Finally, gradients have also been utilized for Out-of-distribution and adversarial detection [5], open set recognition [6], and uncertainty estimation [7].  

References:

  1. M. Prabhushankar, G. Kwon, D. Temel, and G. AlRegib, "Contrastive Explanations in Neural Networks," in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]

  2. G. AlRegib and M. Prabhushankar, "Explanatory Paradigms in Neural Networks," in IEEE Signal Processing Magazine, Special Issue on Explainability in Data Science, Feb. 18 2022. [PDF][Code]

  3. M. Prabhushankar and G. AlRegib, "Extracting Causal Visual Features for Limited Label Classification," in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021. [PDF]

  4. Y. Sun, M. Prabhushankar, and G. AlRegib, "Implicit Saliency in Deep Neural Networks," in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]

  5. J. Lee, M. Prabhushankar, and G. AlRegib, "Gradient-Based Adversarial and Out-of-Distribution Detection," in International Conference on Machine Learning (ICML) Workshop on New Frontiers in Adversarial Machine Learning, Baltimore, MD, Jul. 2022. 

  6. J. Lee and G. AlRegib, "Open-Set Recognition with Gradient-Based Representations," in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021. 

  7. J. Lee and G. AlRegib, "Gradients as a Measure of Uncertainty in Neural Networks," in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020.