---
title: "Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.40328v1"
permalink: "https://feed.astralor.com/items/itm_d540eeaf65d9a31dd919aab2"
published: "2026-10-29T15:56:31.000Z"
score: 54
channel: "ai"
tags: ["Factorized Representations","Concurrent Temporal Variables","Pretrained Language Models","Language","Prior","Across"]
---

# Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models

## At a glance

Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or contextual quantities, but…

## Key points

- Language models often need to maintain multiple contextual variables that evolve independently within a document.
- Prior work has identified structured representations of individual temporal or contextual quantities, but their joint organization remains less understood.
- We study this question using controlled documents with independently phased language-switch and topic-change timers.

## Full text

Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or contextual quantities, but their joint organization remains less understood. We study this question using controlled documents with independently phased language-switch and topic-change timers. Across ten pretrained checkpoints, both timers are linearly decodable from the same hidden states. Removing a timer-associated centroid plane substantially reduces access to its associated variable while having little effect on the concurrent timer, although the planes are nonorthogonal and contain cross-timer information. Separate timer contributions also explain most reproducible variation among joint-state centroids. These signatures persist from 3 to 51 timer values and across Qwen3 checkpoints from 0.6B to 14B. Schedule position affects continuation probabilities, but the tested centroid displacement produces no consistent timer-specific output effect. Together, these results support approximate factorization based on selective accessibility and approximately additive joint-state geometry in overlapping subspaces.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_d540eeaf65d9a31dd919aab2