---
title: "Cognitive-structured Multimodal Agent: A New Paradigm for Long-horizon Multimodal Understanding, Generation, and Editing"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.08497v1"
permalink: "https://feed.astralor.com/items/itm_8a4c877639e13ab1ba8b9111"
published: "2026-07-09T13:55:55.000Z"
score: 79
channel: "tech"
tags: ["arXiv","CMA-Harness","Multimodal Agent","Multimodal Understanding and Generation","Episodic Visual Memory"]
---

# Cognitive-structured Multimodal Agent: A New Paradigm for Long-horizon Multimodal Understanding, Generation, and Editing

## AI commentary

> What stands out most in this paper is the elegant cognitive-architectural solution—replacing full-history attention with external memory and selective retrieval—that improves performance while dramatically reducing inference costs, instead of blindly scaling parameters. For teams building long-context multimodal products, the CMA-Harness framework and the synthetic data generator are highly instructive.

## At a glance

This paper proposes a Cognitive-structured Multimodal Agent that externalizes visual information into Episodic Visual Memory and selectively retrieves relevant episodes, addressing visual token explosion and unreliable cross-turn referencing in existing unified multimodal models during long dialogues. It comprises a Perceptual Abstraction Engine, a Cognitive Retrieval Engine, and a Multimodal Executive Controller, along with a Unified Scenario Engine for generating structured multi-turn conversations with fine-grained retrieval annotations. The 8B model achieves 91.4% retrieval accuracy over 20-turn sessions, halving inference time and surpassing 32B baselines. The CMA-Harness tool-augmented deployment framework is also released.

## Key points

- Proposes a Cognitive-structured Multimodal Agent that uses episodic visual memory and selective retrieval to overcome core bottlenecks in long-horizon multimodal dialogue.
- The 8B model achieves 91.4% retrieval accuracy over 20-turn sessions, reducing inference time from 23.1s to 12.7s, surpassing 32B baselines.
- Builds a Unified Scenario Engine and a long-horizon visual dialogue benchmark to enable reinforcement learning for abstraction and retrieval policy optimization.
- Open-sources the CMA-Harness framework with persistent multimodal memory, web access, and image generation/editing/composition tools.

## Full text

Recent unified multimodal models show a single architecture can jointly perform vision/language understanding and image generation/editing. However, they repeatedly feed all historical visual and textual inputs into a shared context window, limiting long-horizon multimodal dialogue due to visual token explosion and unreliable cross-turn referencing. We propose a Cognitive-structured Multimodal Agent that externalizes visual information into an Episodic Visual Memory and selectively reactivates relevant episodes during reasoning. The agent consists of a Perceptual Abstraction Engine for structured visual abstraction, a Cognitive Retrieval Engine for cross-turn memory retrieval, and a Multimodal Executive Controller for autonomous task inference and action planning. To address the lack of turn-level retrieval supervision in existing datasets, we develop a Unified Scenario Engine that programmatically generates structured multi-turn conversations with fine-grained retrieval annotations, enabling reinforcement learning to optimize abstraction and retrieval policies. We also construct a long-horizon visual-dialogue benchmark stratified by difficulty to evaluate episodic visual recall. Our 8B agent achieves 91.4% retrieval accuracy over 20-turn sessions, surpassing 32B baselines by +8.2% while nearly halving per-turn inference time (23.1s -> 12.7s). We further present the Cognitive-structured Multimodal Agent Harness (CMA-Harness), a tool-augmented deployment of the same cognitive structure integrating persistent multimodal memory, web access, image generation/editing/composition tools, and OpenAI-compatible serving. Structured memory and modular decision-making offer a more scalable, efficient paradigm for long-horizon multimodal agents than monolithic parameter scaling. Code: https://github.com/caseclose/cma-harness ; Project page: https://caseclose.github.io/cma-harness/

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_8a4c877639e13ab1ba8b9111