Estimating Uncertainty with Representation Shifts

 Personnel: Ryan Benkert, Jinsol Lee, Gukyeong Kwon, Mohit Prabhushankar


Goal/Motivation: Accurate uncertainty estimates are essential for trust development in neural network predictions. Algorithms are developed to produce accurate uncertainty scores for different data points

Challenges: Accurate uncertainty estimation remains a major challenge for machine learning research. Neural networks are infamous for poor calibration and produce inconsistent uncertainty estimates for unknown inputs FIXME!!!. Several approaches in uncertainty estimation rely on accurate distance-awareness within the representation space. In other words, they map less confident samples further away from confident samples. Unfortunately, guaranteeing distance awareness is challenging and existing methods result in approximations. For this purpose, the estimation is frequently inaccurate especially when data points are from similar classes.

High Level Description of the Work: Distance-awareness is required only when uncertainty estimates are extracted from the fully trained model directly. For this purpose, we measure uncertainty from model fluctuations or representation shifts. In [1, 2], our team estimates uncertainty by measuring the required model update with gradients. In other words, a model requires a more significant change to learn low confidence data points; a less significant change for high confidence data points. In [3, 4, 5, 6], we measure representation shifts with prediction switches; a second-order uncertainty estimate. Data points with frequently switching predictions are more uncertain than points consistent representations. Empirically, we find our methods result in more fine-grained uncertainty scores and accurate confidence estimates (Figure~1). 

References:

  1. J. Lee and G. AlRegib, "Gradients as a Measure of Uncertainty in Neural Networks," in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020.

  2. G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, "Backpropagated Gradient Representations for Anomaly Detection," in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28 2020. [PDF] [Code]

  3. R. Benkert, M. Prabhushankar, and G. AlRegib, "Reliable Uncertainty Estimation for Seismic Interpretation with Prediction Switches," in International Meeting for Applied Geoscience & Energy (IMAGE), Houston, TX, Aug. 28-Sept. 1 2022.

  4. R. Benkert, O. J. Aribido and G. AlRegib, "Explainable Seismic Neural Networks Using Learning Statistics," in International Meeting for Applied Geoscience & Energy, Denver, CO, 2021.

  5. R. Benkert, O.J. Aribido, and G. AlRegib, "Explaining Deep Models Through Forgettable Learning Dynamics," in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021.

  6. R. Benkert, O.J. Aribido, and G. AlRegib, "Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation," in IEEE Transactions on Geoscience and Remote Sensing (TGRS), May. 12 2022.