---
title: "通过稳定商实现全纯覆盖决策过程中的最小马尔可夫化"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.27132v1"
permalink: "https://feed.astralor.com/items/itm_4d351e6f3fab468733e742c8"
published: "2026-07-29T17:03:35.000Z"
score: 76
channel: "tech"
tags: ["全纯覆盖决策过程","全纯记忆强化学习","稳定商","部分可观测马尔可夫决策过程","马尔可夫化"]
---

# 通过稳定商实现全纯覆盖决策过程中的最小马尔可夫化

## AI 点评

> 这篇论文对POMDP理论有重要推进，特别是对记忆下界的证明非常优雅。对于研究强化学习理论、记忆机制或部分可观测控制的同学尤其值得一读。不过实验部分仅展示了玩具环境，实际部署前还需更多鲁棒性验证。

## 速览

本文针对部分可观测马尔可夫决策过程（POMDP）的子类——全纯覆盖决策过程，提出了最小马尔可夫充分统计量的构造方法。通过定义稳定商（stable quotient），将原始观测抽象为最粗的保持单步奖励和商后继的划分，并证明当前观测与稳定类的组合构成精确的有限马尔可夫状态。在可达性和决策分离条件下，该状态所需记忆单元数量达到理论下限。实验表明，该方法能够从原始状态中提取精确压缩，仅用三个记忆状态即可实现完美的成对顺序预测，超越非神谕基线。

## 关键点

- 提出了稳定商的概念，作为观测的最粗抽象，保持单步奖励和转移结构。
- 证明了当前观测与稳定类的组合构成最小马尔可夫充分统计量，且在特定条件下记忆单元数量下界可达。
- 设计了全纯记忆强化学习框架，通过稳定类表示记忆，利用边传输更新，并可通过校准-重启策略恢复有限MDP保证。
- 实验验证了从原始状态到商状态的精确压缩，仅需三个记忆状态即可完美预测，超越基线方法。

## 全文

An agent acting under partial observability must retain a recursively updateable statistic of history that restores the Markov property, but the smallest such statistic is generally unknown. We characterize this minimal Markov sufficient statistic for holonomy-cover decision processes, a structured POMDP class in which the visible dynamics are Markov and every realized visible transition applies a fixed permutation to a hidden mode. In particular, we construct the stable quotient, the coarsest observation-wise abstraction preserving one-step rewards and quotient successors, and prove that the pair of the current observation and stable class forms an exact finite Markov state. When the current class is correctly initialized, exact class tracking requires exactly the minimal memory symbols, in the sense that under reachability and pairwise decision separation at a maximizing observation, no arbitrary finite-memory controller can use fewer. Under resettable diagnostics, nearest-prototype class inference has exponentially decaying error, and a calibrate-then-restart reduction transfers finite-MDP guarantees to the recovered state. The results enable \emph{Holonomy Memory Reinforcement Learning}. It represents memory by the current stable class, updates it through ordered edge transports, identifies local class coordinates when diagnostics are available, and applies a standard finite-MDP RL backbone after synchronization. Experiments recover an exact compression from raw states to quotient states and achieve perfect paired-order accuracy with three decision-time memory states, matching the quotient oracle and outperforming the non-oracle baselines.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_4d351e6f3fab468733e742c8