The Smartest Know Their Limits: A Novel Semi-Supervised Causal Inference Deep Learning Model for Open Set Recognition

Overview

Given the unrestricted nature of real-world image classification tasks, open-set recognition (OSR), which generalizes conventional closed-set classification to include unseen categories during testing time, has become an important and challenging topic of study. Recent OSR models rely heavily upon large, manually annotated datasets to be effectively trained. However, labeled data is often expensive to collect in practical applications. To address this issue, I propose a novel semi-supervised causal OSR framework, allowing a deep neural network to better learn causal-based attributes by exploiting both labeled and unlabeled image data. Specifically, I adapt the Teacher-Student framework for the generative causal OSR model to make unlabeled data utilization possible, introducing perturbation strategies to define a consistency loss between the outputs of the teacher and student models. In addition, I design a novel counterfactual contrastive loss to further enable the model to learn causal and non-causal attributes using both labeled and unlabeled data, allowing for more reliable counterfactual generation. Extensive experiments across 5 benchmark OSR datasets show that my approach outperforms recent state-of-the-art competitors in terms of accuracy and F1-Scores. Visualizations reveal the generation of reasonable counterfactual images, which indicates the successful disentanglement of causal and non-causal features. Finally, further experiments performed on the ISIC 2019 and Derm7pt skin lesion datasets also demonstrate that my model’s improved performance extends to realistic applications. These contributions in OSR provide promising potential in multiple real-world, non-stationary scenarios, such as medical diagnosis, autonomous driving, speech recognition, etc.

Awards (2)

• Fourth Award of $500 $500
• SAE International: Best Projects

Competition history

• ISEF 2022 Robotics and Intelligent Machines · Entry ROBO010 · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.