S6: Semi-Supervised Self-Supervised Semantic Segmentation

 Collaborators: Moamen Soliman, Charlie Lehman, Ghassan AlRegib

Goal/Motivation: To learn transferable features via self-supervision for a downstream supervised task with limited labeled training data   

Challenges: Deep neural networks are known for their superior performance over traditional machine learning algorithms. In fact, in some applications deep neural networks can achieve super-human-level performance. However, such supremacy comes at the cost of the requirement of huge amounts of labeled examples. In many domain-specific applications, obtaining sufficiently large amounts of labeled data can be expensive, time consuming and impractical. To address the problem of scarcity of data, semi-supervised learning (SSL) techniques are employed. In contrast to fully-supervised learning algorithms, which require all the examples to be labeled, SSL uses unlabeled examples to learn structure about the data, thus reducing the dependency on labeled examples.

High Level Description of the Work: The main contribution of this work is the study of image reconstruction as a form of self-supervision for image segmentation as a main task. The proposed model learns image segmentation in fully-supervised fashion and image reconstruction as a self-supervised auxiliary task. The system encoder-decoder architecture is shown in the figure below. 

It is constrained by the reconstruction branch to learn a representation that can perform segmentation as well as reconstruction. We put forward the hypothesis that the reconstruction branch constrains the deep network representation to be richer since it segments and reconstructs the original image instead of segmenting only. Such arrangement hands over most of the learning from decoders to the encoder since the same encoder representation is now serving two different geometric tasks instead of one. The segmentation performances in both configurations are benchmarked on the Netherlands Offshore F3 block dataset [1] and are shown in the table below.

References

  1. Alaudah, Yazeed, Patrycja Michałowicz, Motaz Alfarraj, and Ghassan AlRegib. "A machine-learning benchmark for facies classification." Interpretation 7, no. 3 (2019): SE175-SE187. [PDF][Code]

  2. M. Soliman, C. Lehman and G. AlRegib, "S6: Semi-Supervised Self-Supervised Semantic Segmentation," 2020 IEEE International Conference on Image Processing (ICIP), 2020, pp. 1861-1865, doi: 10.1109/ICIP40778.2020.9190798. [PDF]