---
title: "HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.41147v1"
permalink: "https://feed.astralor.com/items/itm_f5710350d03d9343a32bde43"
published: "2026-10-30T02:09:34.000Z"
score: 53
channel: "tech"
tags: ["推理效率","RAG","HyperRepair","Recovering Forgotten General","Capabilities","Fine-Tuned Language Models","Fine-tuning","Existing"]
---

# HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models

## 速览

Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it d…

## 关键点

- Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities.
- Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to preserve both.
- We introduce HyperRepair, a framework for recovering general capabilities while retaining domain specialization at only inference cost.

## 全文

Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to preserve both. We introduce HyperRepair, a framework for recovering general capabilities while retaining domain specialization at only inference cost. To make recovery generalizable across models sharing a pretrained backbone, HyperRepair learns a hypernetwork from repairs trained on source models. The hypernetwork uses activation fingerprints that summarize changes from the pretrained backbone to generate a tailored Repairing Side Net (RSN), whose prompt-conditioned gate applies corrections selectively. This enables recovery on models from unseen adaptation domains without target-domain examples or additional optimization. We evaluate HyperRpair across two model families, 12 LoRA adaptation datasets, and 15 general benchmarks. On the default split, HyperRepair achieves 93.4% of the base models' average general score while retaining 99.5% of the fine-tuned models' average domain score.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_f5710350d03d9343a32bde43