Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition

Overview

A critical challenge in Artificial Intelligence is “shortcut learning,” where models rely on superficial shortcuts rather than true causal understanding. A prominent manifestation of this is background bias, where human action recognition models frequently default to background cues rather than human movement and pose to make predictions. For instance, given a video of a person playing violin in a baseball field, the model may predict “playing baseball” rather than “playing violin” because of the background. Background bias has significant ramifications in high-stakes fields such as healthcare, autonomous driving, and law enforcement. If not addressed, it undermines the reliability and safety of AI in real-world environments. This research is the first to conduct a systematic analysis of background bias across state-of-the-art classification models, contrastive text-image models, and Video Large Language Models (VLLMs). My findings reveal that all models exhibit a strong tendency to default to background reasoning. To address this, I develop four novel architectural designs, including dual-branch fusion networks and a dynamic feature reweighting algorithm. By isolating human kinematics through segmentation and dynamically modulating feature importance, I achieve a 3.78% reduction in background bias compared to SOTA models. Moving on to VLLMs, I design an automated iterative prompt system that steers VLLMs toward human-centric reasoning. This optimization framework reduces background reliance by 9.85% over the SOTA baseline. Finally, I target the root of the problem by engineering an augmented counterfactual training data set. By training on mismatched human actions and background scenes in addition to the original dataset, I motivate the models to decouple action from background and achieve 21% reductions in background dependence from the SOTA baseline. My study establishes a critical framework for enhancing AI robustness and equity, ensuring its reliability in high-stakes real-world deployments.

Competition history

• CSEF 2026 Computational Science (Senior Division) · Entry S-07-07

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Computational Sciences projects
• CSEF Computational Sciences projects
• Every CSEF 2026 project