---
title: "Automated Background Swapping: Robustness against Spurious Backgrounds"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2606.32018v1"
permalink: "https://feed.astralor.com/items/itm_f9337e0c56616f26d8d4123a"
published: "2026-06-30T17:50:24.000Z"
score: 55
channel: "ai"
tags: ["AutoBackSwap","Image Classification","Data Augmentation","Robustness"]
---

# Automated Background Swapping: Robustness against Spurious Backgrounds

## At a glance

This paper proposes AutoBackSwap, which uses a secondary network to disentangle foreground and background, inpaint complete backgrounds, and augment training data by swapping various foregrounds with synthesized backgrounds. With only a few hundred patch-wise labeled samples, it can train the secondary network and automatically augment the full dataset. Even when no training sample breaks the spurious correlation, AutoBackSwap effectively reduces classifiers' reliance on backgrounds, consistently outperforming prior methods across tasks.

## Key points

- Uses a secondary network to separate foreground/background and inpaints complete backgrounds for training augmentation.
- Requires only a few hundred patch-wise labeled samples to train the secondary network and augment the entire dataset automatically.
- Effectively improves robustness even when no training sample breaks the spurious background correlation.
- Consistently outperforms prior methods on multiple image classification tasks.

## Full text

Classifiers based on Deep Neural Networks exhibit strong performance across domains, yet can fail catastrophically if they rely on spurious correlations, i.e., features that are predictive of the target label in the training data but are not causally linked and thus fail to generalize. For the vision domain, many such spurious correlations manifest themselves within the background of the image, where only the foreground is predictive of the class label. In this paper, we introduce Automated Background Swapping (AutoBackSwap) to reduce the reliance of classifiers on such spurious backgrounds. AutoBackSwap uses a secondary network to disentangle the foreground and background, followed by infilling to synthesize complete backgrounds, and finally combines different foregrounds and inpainted backgrounds to augment the training data. We find that patch-wise labeling of just a few hundred samples suffices to train the secondary network and automatically augment the full training dataset on challenging image classification tasks. In contrast to many previous methods, AutoBackSwap proves very effective even if there is not a single sample in the training data breaking the spurious correlation. Across a range of image classification tasks with spurious backgrounds, AutoBackSwap consistently outperforms prior methods.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_f9337e0c56616f26d8d4123a