---
title: "Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.06546v1"
permalink: "https://feed.astralor.com/items/itm_3fd2f3b905cffb550ba96919"
published: "2026-07-07T17:52:17.000Z"
score: 81
channel: "ai"
tags: ["Graph Transformer","Graph Convolutional Attention","Graph Denoising","Graph Diffusion","Spectral Graph Theory","Spectral Attention"]
---

# Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion

## AI commentary

> This paper is a must-read for anyone working on graph neural networks and diffusion models. Instead of just empirical tweaks, it reveals a fundamental limitation of linear attention from a spectral perspective and offers an elegant solution. For practitioners, GCA is a plug-and-play attention replacement—especially impressive when combined with DiGress for accelerated inference.

## At a glance

This paper provides a spectral analysis of attention-based graph denoising, showing that linear attention can only learn an average spectral filter, creating a limitation due to spectral diversity across the distribution. The authors propose Spectral Attention and its practical realization, Graph Convolutional Attention (GCA), which performs spectral denoising via graph-filtered queries and keys. Theoretically, GCA matches idealized Spectral Attention under stochastic block models, and the softmax operation is shown to further denoise by projecting noisy eigenvectors onto the clean eigenspace. Empirically, GCA consistently improves graph denoising and diffusion on synthetic and real datasets, and when used with DiGress, it matches standard graph-transformer performance without expensive structural features.

## Key points

- Linear attention can only learn an average spectral denoising filter, failing to adapt to spectral diversity across graph distributions.
- Proposes Spectral Attention and a practical Graph Convolutional Attention (GCA) that performs spectral denoising via graph filtering.
- Theoretically shows GCA matches idealized Spectral Attention under stochastic block models, and softmax further denoises eigenvectors.
- Empirically, GCA improves graph denoising and diffusion, and with PEARL positional encodings avoids explicit eigendecomposition for faster inference.

## Full text

Denoising graphs is a fundamental problem in graph learning and the core operation of graph diffusion models. Attention-based architectures like graph transformers have recently shown promise in denoising graphs. However, our principled understanding of attention-based graph denoising remains limited, making it unclear whether standard attention is the right mechanism for this task. Here we show that, under a denoising objective, linear attention is suboptimal and can only learn an average spectral denoising filter over the training distribution. This creates a fundamental limitation as graphs often vary spectrally across the distribution. To overcome this limitation, we introduce Spectral Attention, which directly utilizes the input graph spectrum and provably outperforms linear attention by a margin governed by the spectral diversity of the distribution. We then derive Graph Convolutional Attention (GCA), a practical and permutation-equivariant realization of this idea that implements spectral denoising through graph-filtered queries and keys. For stochastic block models, GCA provably matches the idealized Spectral Attention mechanism. We further show that the softmax operation, that follows the attention, provides additional denoising by approximately projecting noisy eigenvectors onto the clean eigenspace. Empirically, replacing linear attention with GCA consistently improves graph denoising and diffusion on synthetic and real datasets, with gains strongly correlated with spectral diversity. In DiGress, GCA matches standard graph-transformer performance without computing expensive structural features, and when combined with the recently proposed PEARL positional encodings, avoids explicit eigendecomposition computations resulting in faster inference without degrading quality. The code can be found here: github.com/shervinkhalafi/graph_conv_att

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_3fd2f3b905cffb550ba96919