---
title: "CARE: Confidence-Adaptive Routing for Mixture-of-Experts LoRA"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.26052v1"
permalink: "https://feed.astralor.com/items/itm_e7f0b97bead291fd9ce401dc"
published: "2026-07-28T17:59:16.000Z"
score: 81
channel: "tech"
tags: ["CARE","LoRA","MoE","Uncertainty Routing","Nucleus Sampling"]
---

# CARE: Confidence-Adaptive Routing for Mixture-of-Experts LoRA

## AI commentary

> This paper is particularly noteworthy for solving a long-overlooked fixed-k efficiency problem in MoE-LoRA with a minimal drop-in solution. For researchers working on model compression and efficient inference, CARE's nucleus-style routing can be easily transferred to other MoE architectures; for deployment engineers, it can be directly integrated into existing LoRA pipelines to reduce compute cost. Although the experiments do not explore larger expert sets, the theoretical analysis and novelty will inspire follow-up work.

## At a glance

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts k, wasting resources on easy tokens and under-serving hard ones. This paper introduces CARE (Confidence-Adaptive Routing of Experts), which uses the router's output distribution as a per-token uncertainty signal and activates experts in a nucleus fashion until cumulative mass reaches a threshold, with a budget thermostat to match target average expert count. On LLaMA-3.1-8B and Qwen2.5-7B across commonsense, math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches k=4 baseline while activating fewer experts; confidence and disagreement signals also improve out-of-distribution detection.

## Key points

- Observes that the router's output distribution is itself a per-token uncertainty signal: peaked mass indicates confidence, flat indicates ambiguity.
- Proposes nucleus-style activation of experts, stopping when cumulative router weight reaches a threshold, with small extension when admitted experts disagree.
- Introduces a budget thermostat to calibrate the threshold so average active expert count matches any target, without extra parameters.
- On LLaMA-3.1-8B and Qwen2.5-7B across diverse tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches k=4 baseline with fewer experts.

## Full text

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts $k$. Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones. We observe that the router's output distribution is already a per-token uncertainty signal: peaked mass indicates confidence, while a flat distribution indicates ambiguity. We introduce CARE (Confidence-Adaptive Routing of Experts), which admits experts in a nucleus fashion. Experts are activated in decreasing router weight until their cumulative mass reaches a threshold, with a small extension when the admitted experts disagree. A budget thermostat calibrates the threshold so that the average number of active experts matches any target. CARE is a drop-in, single-forward-pass rule with no extra parameters. Across eight commonsense benchmarks on LLaMA-3.1-8B and Qwen2.5-7B, as well as math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches the fixed-k=4 baseline while activating fewer experts. The same confidence and disagreement signals also improve out-of-distribution detection over MSP, entropy, and multi-pass proxies. We support the design with nucleus fidelity, budget optimality, and an epistemic reading of disagreement, and we release code.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_e7f0b97bead291fd9ce401dc