---
title: "Learning Noise Optimization for Reward Alignment in Few-Step Generative Models"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2611.06255v1"
permalink: "https://feed.astralor.com/items/itm_652f1b37969f54eccb99f961"
published: "2026-11-04T10:06:32.000Z"
score: 58
channel: "tech"
tags: ["推理效率","安全","Learning Noise Optimization","Reward Alignment","Few-Step Generative Models","Reward","Initial","LeNO"]
---

# Learning Noise Optimization for Reward Alignment in Few-Step Generative Models

## At a glance

Reward alignment for diffusion- and flow-based generative models is often formulated over stochastic trajectories, making it less suitable for few-step deterministic generators. Initial noise optimization provides a tra…

## Key points

- Reward alignment for diffusion- and flow-based generative models is often formulated over stochastic trajectories, making it less suitable for few-step deterministic generators.
- Initial noise optimization provides a trajectory-free alternative, but existing gradient-free methods require costly per-sample search at inference.
- We introduce LeNO (Learning Noise Optimization), which amortizes reward-guided initial noise optimization into a learned residual transport while keeping the pretrained generator frozen.

## Full text

Reward alignment for diffusion- and flow-based generative models is often formulated over stochastic trajectories, making it less suitable for few-step deterministic generators. Initial noise optimization provides a trajectory-free alternative, but existing gradient-free methods require costly per-sample search at inference. We introduce LeNO (Learning Noise Optimization), which amortizes reward-guided initial noise optimization into a learned residual transport while keeping the pretrained generator frozen. LeNO learns an anchor-conditioned distribution over noise corrections using Local GRPO and scalar rewards, eliminating iterative test-time optimization. For Gaussian initial noise with norm-preserving projection, we derive a tractable upper bound on the KL divergence between the corrected noise distribution and the original prior, providing a principled regularization objective. We evaluate LeNO on few-step text-to-image and protein structure generation, including expensive black-box designability rewards. Across both domains, LeNO improves target rewards with a single noise correction at inference while largely preserving generation quality and diversity.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_652f1b37969f54eccb99f961