---
title: "AVA-Encoder：面向智能体的视频表征学习框架"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2608.12313v1"
permalink: "https://feed.astralor.com/items/itm_1c97be6b6bc9e9853225932a"
published: "2026-08-12T17:58:02.000Z"
score: 78
channel: "ai"
tags: ["arXiv","AVA-Encoder","智能体","知识图谱","视频表征学习"]
---

# AVA-Encoder：面向智能体的视频表征学习框架

## AI 点评

> 这篇论文最值得关注的是把知识图谱当作智能体与视频之间的中间语言，绕开了端到端黑盒的局限。如果你在做视频 Agent 或可控生成，这个框架和数据集值得一试。

## 速览

论文提出 AVA-Encoder，一种智能体原生视频表征学习框架，通过将视频编码为知识图谱表示并重构回视频，实现层次化结构存储与可编辑资产层。其独创的文本梯度优化机制使智能体能够以自然语言指令迭代改进编码策略。实验显示，AVA-Encoder 较最强外部基线提升 20.7 个百分点，且策略级编码器在减少 74.3% 系统提示词的情况下仍超越人工调优方案。该框架还发布了首个高质量电影级知识图谱数据集。

## 关键点

- AVA-Encoder 将视频转为知识图谱表示，再重构回视频，形成智能体可理解、可查询、可编辑的闭环。
- 提出文本梯度优化框架：以重构差异生成自然语言更新方向，支持外层策略与内层测试时精调。
- 在基准上超过最强外部基线 20.7%，策略级编码器用 74.3% 更少提示词优于人工调优。
- 开源完整框架、可靠基准及首个高质量电影知识图谱数据集。

## 全文

Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-grade videos. A key challenge is the absence of a structured video representation that is both faithful to film content and directly usable for agentic reasoning and manipulation. To address the challenge, we propose the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agentic auto-encoding. AVA-Encoder transforms a video into a knowledge graph (KG) representation and then reconstructs it back into video. Its hierarchy and state nodes store structured text, while a linked asset layer holds generated images, audio, and video. Typed edges preserve the relations between these text descriptions and assets in a form that agents can easily understand, query, and edit. The video reconstruction differences drive a textual-gradient optimization framework, which expresses evaluation feedback as natural-language update directions for Data-Independent Encoding Policy Pseudo-Training in the outer loop and optional Data-Dependent KG Representation Refinement in the test-time inner loop. Extensive experiments show that AVA-Encoder improves by 20.7 percentage points over the strongest external baseline. In the controlled policy-only setting, its pseudo-trained shot-level Agentic Video Encoder policy also outperforms a carefully human-tuned policy while using 74.3% fewer system-prompt tokens. We release the complete AVA-Encoder framework, a reliable agentic video reconstruction benchmark, and the first dataset of high-quality film KG representations.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_1c97be6b6bc9e9853225932a