Seismic Risk Assessment with Uncertainty Estimation

 Personnel: Ryan Benkert, Jinsol Lee, Mohit Prabhushankar

Goal/Motivation: Neural networks in seismic interpretation suffer under poor calibration and limited annotated training data. Accurate uncertainty estimation algorithms are crucial to ensure risk analysis in model predictions.

Challenges: In seismic interpretation, experts are required for the correct annotation of subsurface volumes. For this purpose, seismic annotations are scarce, costly, and frequently ambiguous as disagreement among experts is common. As a result, deep neural networks are poorly calibrated and are overconfident in inaccurate predictions. Therefore, uncertainty estimation plays a crucial role for trust, risk assessment, or model analysis. Unfortunately, several approaches rely on distance-awareness which is especially challenging to guarantee when limited data is available. For this purpose, the estimation is frequently inaccurate especially when data points are from similar classes.

High Level Description of the Work: Distance-awareness is required only when uncertainty estimates are extracted from the fully trained model directly. For this purpose, we measure uncertainty from model fluctuations or representation shifts. In [1, 2], our team estimates uncertainty by measuring the required model update with gradients. In other words, a model requires a more significant change to learn low confidence data points; a less significant change for high confidence data points. In [3, 4, 5, 6], we measure representation shifts with prediction switches; a second-order uncertainty estimate. Data points with frequently switching predictions are more uncertain than points consistent representations. Empirically, we find our methods result in more fine-grained uncertainty scores and accurate confidence estimates (Figure~1). 

References:

  1. J. Lee and G. AlRegib, "Gradients as a Measure of Uncertainty in Neural Networks," in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020.

  2. G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, "Backpropagated Gradient Representations for Anomaly Detection," in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28 2020. [PDF] [Code]

  3. R. Benkert, M. Prabhushankar, and G. AlRegib, "Reliable Uncertainty Estimation for Seismic Interpretation with Prediction Switches," in International Meeting for Applied Geoscience & Energy (IMAGE), Houston, TX, Aug. 28-Sept. 1 2022.

  4. R. Benkert, O. J. Aribido and G. AlRegib, "Explainable Seismic Neural Networks Using Learning Statistics," in International Meeting for Applied Geoscience & Energy, Denver, CO, 2021.

  5. R. Benkert, O.J. Aribido, and G. AlRegib, "Explaining Deep Models Through Forgettable Learning Dynamics," in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021.

  6. R. Benkert, O.J. Aribido, and G. AlRegib, "Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation," in IEEE Transactions on Geoscience and Remote Sensing (TGRS), May. 12 2022.