---
title: "从表达到样本复杂度：通过C-RASP理解Transformer的窄教师模型"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.11760v1"
permalink: "https://feed.astralor.com/items/itm_8eb391d1d7892aba1bf2cca9"
published: "2026-07-13T16:22:36.000Z"
score: 67
channel: "ai"
tags: ["C-RASP","Transformer","大型语言模型","损失景观","样本复杂度","表达能力"]
---

# 从表达到样本复杂度：通过C-RASP理解Transformer的窄教师模型

## AI 点评

> 这篇论文值得关注，因为它首次将表达能力与样本复杂度这两条理论轨迹合并，对从事LLM理论研究的学者尤其重要。虽然结论还比较初步，但它为后续工作提供了清晰的数学框架，有望推动深度学习理论突破。

## 速览

该论文通过C-RASP框架研究Transformer的表达能力和样本复杂度。作者发现，虽然已有大量工作分析注意力模型的表达能力，但关于这些解决方案的可学习性研究很少。本文基于损失景观分析，提出了学习C-RASP构造的初步样本复杂度界限。这对于理解大型语言模型的能力和局限性具有重要理论意义。

## 关键点

- 提出了C-RASP框架下Transformer学习样本复杂度的初步界限
- 基于损失景观分析，填补了表达能力与可学习性之间的理论空白
- 为理解大型语言模型的能力边界提供了新的理论工具

## 全文

A theoretical understanding of Transformers is crucial to better understand the capacities and limitations of large language models (LLMs). There is much work analyzing the expressivity of attention-based models. By proposing handcrafted weights or using computational complexity arguments, a large amount of past theoretical works have sought to characterize which tasks are and which are not in the hypothesis class of Transformer models. However, little work investigates the learnability of such solutions. In this work, we make progress towards this goal. Inspired by recent loss landscape analysis work, we propose preliminary sample complexity bounds for learning C-RASP constructions with Transformers.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_8eb391d1d7892aba1bf2cca9