---
title: "Calibration of Large Model Quantization based on HiFloat4 Format"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2610.41112v1"
permalink: "https://feed.astralor.com/items/itm_e6800eee0d43d2e1413eb1fa"
published: "2026-10-30T01:10:51.000Z"
score: 55
channel: "tech"
tags: ["LLM","OpenAI","推理效率","Calibration","Large Model Quantization","HiFloat4 Format Quantization","Large Language Models","LLMs"]
---

# Calibration of Large Model Quantization based on HiFloat4 Format

## 速览

Quantization of Large Language Models (LLMs) has be- coming increasingly important for reducing the computa- tional costs and resource requirements for practical deploy- ment.Growinginterestinthisareahasmotivatedthecomm…

## 关键点

- Quantization of Large Language Models (LLMs) has be- coming increasingly important for reducing the computa- tional costs and resource requirements for practical deploy- ment.Growinginterestinthisareahasmotivatedthecomm…
- Recently, HiFloat4 (HiF4) has emerged as a competitive alternative which adopts a Block Floating-Point(BFP)formatwithathree-levelscalinghierar- chy, thereby achieving a significantly larger dynamic range.
- While quantization can substantially reduce memory usage and accelerate inference, it frequently introduces accuracy degradation.

## 全文

Quantization of Large Language Models (LLMs) has be- coming increasingly important for reducing the computa- tional costs and resource requirements for practical deploy- ment.Growinginterestinthisareahasmotivatedthecommu- nity to move beyond 8-bit quantization towards 4-bit preci- sion, leading to the development of customized 4-bit formats such as MXFP4 and NVFP4. Recently, HiFloat4 (HiF4) has emerged as a competitive alternative which adopts a Block Floating-Point(BFP)formatwithathree-levelscalinghierar- chy, thereby achieving a significantly larger dynamic range. While quantization can substantially reduce memory usage and accelerate inference, it frequently introduces accuracy degradation. Calibration is therefore essential for narrowing theperformancegapbetweenlow-precisionquantizedmodels and their full-precision counterparts. Nevertheless, existing calibration methods are mainly designed and validated us- ing integer-precision formats, such as INT4, lacking support for HiF4 due to its hierarchical BFP structure. In this paper, weinvestigatethreecalibrationapproaches,namelyHiGPTQ, HiAutoRound, and HiFlatQuant, to further enhance model quantizationbasedontheHiF4format.Extensiveexperiments demonstratethatallthreeproposedapproachesimproveinfer- ence accuracy compared with direct HiF4 conversion, reduc- ing the gap between HiF4 and BF16 to less than 1% and out- performing the state-of-the-art NVFP4 format across various models and downstream tasks.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_e6800eee0d43d2e1413eb1fa