---
title: "Editing Large Language Models with Geometry-Aware Regularization"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.06185v1"
permalink: "https://feed.astralor.com/items/itm_9e1b416265f7d2aed4fb8977"
published: "2026-11-04T09:02:10.000Z"
score: 59
channel: "tech"
tags: ["LLM","OpenAI","Editing Large Language","Models","Geometry-Aware Regularization Knowledge","However","Knowledge","GEdit"]
---

# Editing Large Language Models with Geometry-Aware Regularization

## At a glance

Knowledge editing has become a key technique for updating knowledge in large language models. However, existing methods neglect the geometric structure of the knowledge representation space, leading to two key limitatio…

## Key points

- Knowledge editing has become a key technique for updating knowledge in large language models.
- However, existing methods neglect the geometric structure of the knowledge representation space, leading to two key limitations: they cannot distinguish which pre-trained knowledge should be preferentially preserved (co…
- non-core) nor which new edits are inherently difficult (hard vs.

## Full text

Knowledge editing has become a key technique for updating knowledge in large language models. However, existing methods neglect the geometric structure of the knowledge representation space, leading to two key limitations: they cannot distinguish which pre-trained knowledge should be preferentially preserved (core vs. non-core) nor which new edits are inherently difficult (hard vs. easy). To address this, we propose GEdit, a geometry-aware framework built on a Low-Rank Plus Diagonal (LR+D) factor model that captures the intrinsic low-rank structure of Transformer FFN hidden representations. GEdit features two complementary mechanisms. First, the low-rank regularization applies targeted protection on a low-rank core knowledge subspace. Second, the adaptive regularization dynamically adjusts the regularization strength based on each edit request's editability: relaxing constraints for hard edits to ensure success, and tightening them for easy edits to better preserve pretrained knowledge. Extensive experiments on GPT2-XL, GPT-J, and LLaMA-3-8B under large-scale editing scenarios, including both mini-batch and fully sequential editing settings with up to 10,000 continuous updates, show that GEdit consistently and significantly outperforms state-of-the-art baselines in both editing success and knowledge preservation.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_9e1b416265f7d2aed4fb8977