---
title: "Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.36794v1"
permalink: "https://feed.astralor.com/items/itm_48d6ee1404a5d6f4f3d0377f"
published: "2026-10-27T07:58:39.000Z"
score: 58
channel: "ai"
tags: ["LLM","AI Agent","多模态","Select-TTT","Query-Conditioned Selective Fast","Weight Updates","In-Place Test-Time Training","Test-time"]
---

# Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training

## 速览

Test-time training (TTT) enables a deployed language model to internalize incoming context on-the-fly into a small set of fast-weight parameters, rather than merely treating it as transient context for attention. In-Pla…

## 关键点

- Test-time training (TTT) enables a deployed language model to internalize incoming context on-the-fly into a small set of fast-weight parameters, rather than merely treating it as transient context for attention.
- In-Place TTT recently made this idea practical for large language models by repurposing the down-projection of standard MLP blocks as fast weights.
- However, we find that its benefit largely disappears on realistic web long-context question answering.

## 全文

Test-time training (TTT) enables a deployed language model to internalize incoming context on-the-fly into a small set of fast-weight parameters, rather than merely treating it as transient context for attention. In-Place TTT recently made this idea practical for large language models by repurposing the down-projection of standard MLP blocks as fast weights. However, we find that its benefit largely disappears on realistic web long-context question answering. We attribute this to In-Place TTT's uniform write schedule: every chunk is written with equal strength, so irrelevant content dilutes query-relevant information in fast-weight memory. Building on this insight, we propose Select-TTT, a model-agnostic, query-conditioned rule for In-Place TTT that scales each chunk's write strength by its relevance to the user query. This relevance signal requires no supervision and no new parameters: it is read directly from the model's frozen output head, as the probability mass that a chunk's hidden states assign to the query tokens. A temperature-scaled softmax turns these scores into learning-rate multipliers, leaving the chunk-wise, context-parallel computation of In-Place TTT intact. Select-TTT consistently improves long-context question answering on two benchmarks and two backbones. These findings highlight the value of query-conditioned fast weight writes for practical test-time training on long-context question answering, which is also potentially promising for long-horizon agent tasks involving multiple user queries over evolving contexts.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_48d6ee1404a5d6f4f3d0377f