---
title: "BiSCo-LLM：面向极端低位大语言模型压缩的无查找二进制球形编码"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.08643v1"
permalink: "https://feed.astralor.com/items/itm_48362c9ed9e9bb9f7cf7a716"
published: "2026-07-09T16:17:10.000Z"
score: 77
channel: "tech"
tags: ["arXiv","BiSCo-LLM","二进制球形编码","低位量化","大语言模型","模型压缩"]
---

# BiSCo-LLM：面向极端低位大语言模型压缩的无查找二进制球形编码

## AI 点评

> 这篇论文在极低位权重复制上做出了很有创意的尝试，将符号流作为主载荷的设计跳出了传统码本思维，在2比特以下区域效果显著。对从事模型部署、边缘端AI的工程师和研究者尤其值得关注，可能成为未来低比特LLM压缩的新基准。

## 速览

本文提出BiSCo-LLM，一种无需码本的二元球形编码框架，用于大语言模型（LLM）的极端低位权重压缩。该方法将局部权重块映射到单位超球面并二值化为紧凑球面码，主载荷仅含位打包符号流；通过残差BSQ阶段编码重构误差，实现无存储码本的率失真路径；采用分模块恢复蒸馏减少局部重建与模型行为间的失配。8位保护通道作为敏感通道的辅助稳定机制，总存储预算包括二进制码、神经解码器、保护通道载荷、LoRA适配器和元数据。实验表明，在极低比特率下显著优于现有量化与向量量化方法。

## 关键点

- 无需显式码本的二进制球形编码，主载荷仅为符号位流。
- 残差BSQ阶段提供无存储码本的率失真路径。
- 分模块恢复蒸馏减少局部重建与整体模型行为失配。
- 8位保护通道作为敏感通道辅助稳定机制，额外计入总预算。

## 全文

Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainly follow two directions. Scalar or group-wise quantization is simple and compatible with efficient low-precision kernels, but its representation capacity becomes limited when the target budget approaches 2 bits per weight. Vector-quantized weight compression provides a richer block-level representation, but usually introduces explicit codebooks, index lookup, and additional storage accounting. This paper presents BiSCo-LLM, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression. The core pipeline is built on three components. First, local weight chunks are mapped onto a unit hypersphere and binarized into compact spherical codes, so that the main payload is a bit-packed sign stream rather than explicit VQ centroids. Second, a residual BSQ stage encodes the reconstruction error left by the base spherical codec, providing an explicit rate-distortion path without stored codebooks. Third, category-wise recovery distillation is performed after replacing each Transformer module category, reducing the mismatch between local weight reconstruction and assembled model behavior. A small 8-bit protected-channel path is used as an auxiliary stabilization mechanism for sensitive channels and is counted separately from the BSQ payload. The reported storage budget includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_48362c9ed9e9bb9f7cf7a716