---
title: "Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.27132v1"
permalink: "https://feed.astralor.com/items/itm_4d351e6f3fab468733e742c8"
published: "2026-07-29T17:03:35.000Z"
score: 76
channel: "tech"
tags: ["Holonomy-Cover Decision Processes","Holonomy Memory Reinforcement Learning","Stable Quotient","Partially Observable Markov Decision Process","Markovization"]
---

# Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes

## AI commentary

> This paper makes a significant theoretical advancement in POMDPs, especially the elegant proof of memory lower bounds. It is particularly valuable for researchers in reinforcement learning theory, memory mechanisms, or partially observable control. However, the experiments are limited to toy environments; more robustness validation is needed before deployment.

## At a glance

This paper addresses the minimal Markov sufficient statistic for a structured POMDP subclass called holonomy-cover decision processes. It constructs the stable quotient, the coarsest observation-wise abstraction that preserves one-step rewards and quotient successors, and proves that the pair of current observation and stable class forms an exact finite Markov state. Under reachability and pairwise decision separation, this state requires the minimal number of memory symbols. Experiments show exact compression from raw states to quotient states, achieving perfect paired-order accuracy with three memory states, outperforming non-oracle baselines.

## Key points

- Introduces the stable quotient, the coarsest observation abstraction that preserves one-step rewards and transition structure.
- Proves that the pair of current observation and stable class forms a minimal Markov sufficient statistic, with a lower bound on memory symbols under reachability and decision separation.
- Proposes Holonomy Memory Reinforcement Learning, representing memory via stable class, updating via ordered edge transports, and recovering finite-MDP guarantees through calibrate-restart.
- Experiments demonstrate exact compression from raw states to quotient states, achieving perfect paired-order accuracy with only three memory states, surpassing baselines.

## Full text

An agent acting under partial observability must retain a recursively updateable statistic of history that restores the Markov property, but the smallest such statistic is generally unknown. We characterize this minimal Markov sufficient statistic for holonomy-cover decision processes, a structured POMDP class in which the visible dynamics are Markov and every realized visible transition applies a fixed permutation to a hidden mode. In particular, we construct the stable quotient, the coarsest observation-wise abstraction preserving one-step rewards and quotient successors, and prove that the pair of the current observation and stable class forms an exact finite Markov state. When the current class is correctly initialized, exact class tracking requires exactly the minimal memory symbols, in the sense that under reachability and pairwise decision separation at a maximizing observation, no arbitrary finite-memory controller can use fewer. Under resettable diagnostics, nearest-prototype class inference has exponentially decaying error, and a calibrate-then-restart reduction transfers finite-MDP guarantees to the recovered state. The results enable \emph{Holonomy Memory Reinforcement Learning}. It represents memory by the current stable class, updates it through ordered edge transports, identifies local class coordinates when diagnostics are available, and applies a standard finite-MDP RL backbone after synchronization. Experiments recover an exact compression from raw states to quotient states and achieve perfect paired-order accuracy with three decision-time memory states, matching the quotient oracle and outperforming the non-oracle baselines.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_4d351e6f3fab468733e742c8