OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics

Personnel: Kiran Kokilepersaud, Mohit Prabhushankar, Yash Yee Logan

Goal: The medical domain has a wide range of perspectives that are not accounted for. This includes biomarker information, clinical labels, time-series treatment prediction, and various imaging modalities. Can we create a dataset that is able to highlight the interactions between these diverse modalities?

Challenges: The challenge lies in the interaction between these modalities as well as the target tasks intended for the medical domain. For example, reasonable questions are what relationships exist between biomarkers and clinical data, how do these relationships change over time, and how do these relationships manifest within the context of the associated imaging data we have at our disposal? Each one of these questions comes with their own set of associated problems such as time-series model selection, how to make use of the unlabeled pool of data, and appropriate selection of loss functions to fuse information from multiple modalities.  

Our Work:

[1] is the resulting dataset 

Datasets Utilized:

  1. Prabhushankar, M., Kokilepersaud, K. P., Logan, Y. Y., Corona, S. T., AlRegib, G., & Wykoff, C. “OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics." in Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, LA, Nov. 29 - Dec. 1 2022 [PDF] [CODE]