---
title: "CARE：基于置信度自适应路由的混合专家LoRA方法"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26052v1"
permalink: "https://feed.astralor.com/items/itm_e7f0b97bead291fd9ce401dc"
published: "2026-07-28T17:59:16.000Z"
score: 81
channel: "tech"
tags: ["CARE","LoRA","MoE","不确定性路由","核采样"]
---

# CARE：基于置信度自适应路由的混合专家LoRA方法

## AI 点评

> 这篇论文特别值得关注，因为它用一个极简的即插即用方案解决了MoE-LoRA中长期被忽视的固定k效率问题。对于从事模型压缩和高效推理的研究者来说，CARE的核采样思路可轻松迁移到其他MoE架构；而对部署工程师，它能直接注入现有LoRA流程降低计算成本。虽然实验没有探索更大的专家集，但理论分析和方法新颖性足以激发后续工作。

## 速览

针对混合专家（MoE）低秩适应（LoRA）中固定专家数量k导致的资源浪费和困难token服务不足问题，本文提出CARE方法，利用路由器输出分布作为不确定性信号，以核采样方式逐步激活专家，直到累积质量达到阈值，并设置预算恒温器控制平均专家数。在LLaMA-3.1-8B和Qwen2.5-7B上的常识推理、数学、代码等任务中，CARE在相同计算量下优于固定top-k MoE-LoRA，且使用更少专家达到固定k=4基线性能；同时置信度与分歧信号改善了分布外检测。

## 关键点

- 观察到路由器输出分布本身就是逐token的不确定性信号，峰值指示置信度，平坦指示模糊性。
- 提出核采样方式激活专家，当累积路由器权重达到阈值时停止，若已激活专家分歧则轻微扩展。
- 设置预算恒温器校准阈值，使平均活跃专家数匹配目标，无需额外参数。
- 在LLaMA-3.1-8B和Qwen2.5-7B上超常任务，CARE在相同计算量下超越固定top-k MoE-LoRA，且用更少专家达到k=4基线。

## 全文

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts $k$. Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones. We observe that the router's output distribution is already a per-token uncertainty signal: peaked mass indicates confidence, while a flat distribution indicates ambiguity. We introduce CARE (Confidence-Adaptive Routing of Experts), which admits experts in a nucleus fashion. Experts are activated in decreasing router weight until their cumulative mass reaches a threshold, with a small extension when the admitted experts disagree. A budget thermostat calibrates the threshold so that the average number of active experts matches any target. CARE is a drop-in, single-forward-pass rule with no extra parameters. Across eight commonsense benchmarks on LLaMA-3.1-8B and Qwen2.5-7B, as well as math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches the fixed-k=4 baseline while activating fewer experts. The same confidence and disagreement signals also improve out-of-distribution detection over MSP, entropy, and multi-pass proxies. We support the design with nucleus fidelity, budget optimality, and an epistemic reading of disagreement, and we release code.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_e7f0b97bead291fd9ce401dc