---
title: "Robust LLM Data Selection via Content-Invariant Gradient Attribution"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.36755v1"
permalink: "https://feed.astralor.com/items/itm_6d3b01320a91ebc951eb1cb5"
published: "2026-10-27T07:12:01.000Z"
score: 57
channel: "tech"
tags: ["LLM","推理效率","RAG","Robust LLM Data","Selection","Content-Invariant Gradient Attribution","Gradient-based","LLMs"]
---

# Robust LLM Data Selection via Content-Invariant Gradient Attribution

## At a glance

Gradient-based data attribution has become an important tool for high-quality data selection in the instruction fine-tuning of large language models (LLMs), as it estimates the contribution of training examples through …

## Key points

- Gradient-based data attribution has become an important tool for high-quality data selection in the instruction fine-tuning of large language models (LLMs), as it estimates the contribution of training examples through …
- However, we find that existing global gradient attribution methods suffer from surface-form bias: minor changes to semantically irrelevant details, such as wording or formatting, can produce highly inconsistent attribut…
- To address this issue, we propose Content-Invariant Gradient Attribution (CIGA), a robust framework for instruction data selection.

## Full text

Gradient-based data attribution has become an important tool for high-quality data selection in the instruction fine-tuning of large language models (LLMs), as it estimates the contribution of training examples through their influence on model optimization. However, we find that existing global gradient attribution methods suffer from surface-form bias: minor changes to semantically irrelevant details, such as wording or formatting, can produce highly inconsistent attribution scores even when the underlying content remains unchanged. To address this issue, we propose Content-Invariant Gradient Attribution (CIGA), a robust framework for instruction data selection. CIGA first identifies layers whose gradients are both influential for attribution and stable under semantics-preserving surface-form perturbations, thereby reducing formatting noise in gradient features. It then dynamically fuses these attribution signals with semantic representations of the input, combining optimization-level influence with content-level information. Extensive experiments across multiple benchmarks show that CIGA produces more stable rankings under surface-form perturbations while preserving downstream capabilities and improving average performance. CIGA also reduces attribution cost, offering a robust and efficient approach to data selection for LLM instruction tuning.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_6d3b01320a91ebc951eb1cb5