---
title: "Where It Breaks: Diagnosing Perception, Memory, and Decision Failures of a Vision-Language Agent in Pac-Man"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.06704v1"
permalink: "https://feed.astralor.com/items/itm_01038fdcf08b3464189bd310"
published: "2026-11-04T16:04:57.000Z"
score: 58
channel: "ai"
tags: ["AI Agent","多模态","Where","Breaks","Diagnosing Perception","Memory","Decision Failures","Vision-Language Agent"]
---

# Where It Breaks: Diagnosing Perception, Memory, and Decision Failures of a Vision-Language Agent in Pac-Man

## At a glance

Training an agent to play Pac-Man with only up, down, left and right actions raises a basic question: when performance is poor, what should be improved? Difficulty can arise from reading the screen or from choosing acti…

## Key points

- Training an agent to play Pac-Man with only up, down, left and right actions raises a basic question: when performance is poor, what should be improved?
- Difficulty can arise from reading the screen or from choosing actions over a long sequence.
- We examine these problems in a Qwen3.5-9B-based agent through separate perception checks and primitive-policy reinforcement-learning studies.

## Full text

Training an agent to play Pac-Man with only up, down, left and right actions raises a basic question: when performance is poor, what should be improved? Difficulty can arise from reading the screen or from choosing actions over a long sequence. We examine these problems in a Qwen3.5-9B-based agent through separate perception checks and primitive-policy reinforcement-learning studies. In the policy study, which uses textual boards to remove image transcription from the decision interface, completion on eight held-out mazes increased from 0% before this optimization to 80.2% at a selected checkpoint, with three extra lives. The perception checks show that accurate static screenshot reading can coexist with errors during interaction. A separate policy comparison also shows that further optimization can accompany worse action choices. Together, these findings motivate concrete checks for post-training: evaluate perception in use, distinguish training success from full-game validation, and examine decisions and later performance rather than relying on one selected score. The study provides empirical lessons for improving future agent training and evaluation.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_01038fdcf08b3464189bd310