---
title: "顺序控制交互式多粒子流映射用于在线反馈驱动搜索"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.01144v1"
permalink: "https://feed.astralor.com/items/itm_39ac28e508d01204e4b89145"
published: "2026-07-01T16:27:17.000Z"
score: 59
channel: "ai"
tags: ["IMPFM","在线学习","多粒子系统","探索-利用权衡","流映射"]
---

# 顺序控制交互式多粒子流映射用于在线反馈驱动搜索

## 速览

该论文提出IMPFM框架，通过多粒子交互流映射实现高效在线反馈驱动搜索，解决偏好未知场景下的广泛探索问题。框架采用后验样本共享机制和探索-利用重加权策略，理论证明其能驱动多粒子系统向KL倾斜目标分布收敛，防止模式坍塌。在多种搜索和对齐任务中，IMPFM显著优于现有基线方法。

## 关键点

- 提出IMPFM框架，结合流映射与序列控制的多粒子动力学，实现全局探索与反馈驱动对齐
- 创新性地引入后验样本共享机制，利用集体后验样本纠正个体漂移，提升样本利用率
- 理论证明多粒子交互Feynman-Kac校正器能使系统渐进收敛至KL倾斜分布，保证多样性

## 全文

While generative models have enabled training-free reward alignment, current methods typically excel in local exploration within narrow regions of the underlying distribution. These approaches struggle when preferences are unknown a priori and only revealed through sequential feedback-a scenario demanding broad exploration to uncover high-utility regions. To address this, we propose Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for sample-efficient online feedback-driven search. IMPFM progressively transports a group of interactive particles toward the target distribution, maintaining the broad coverage essential for heterogeneous preference alignment. IMPFM introduces a principled and efficient posterior sample sharing mechanism across particles powered by flow maps. By correcting individual particle drift with the collective posterior samples of the entire ensemble at each resampling step, the framework maximizes sample utility to enable global exploration while actively mitigating reward over-optimization, typical of standard control frameworks. Paired with a principled exploration-exploitation reweighting mechanism involving multi-particle interaction, this sequentially corrected multi-particle dynamics explicitly preserves structural diversity and overcomes the weight degeneracy inherent to standard SMC samplers. Crucially, we prove that the resulting sampling framework yields a multi-particle interaction-aware Feynman-Kac corrector that progressively steers the multi-particle system toward a KL-tilted target distribution, facilitating global exploration and preventing mode collapse. Extensive empirical evaluations and rigorous ablations across diverse search and alignment tasks confirm the efficacy of IMPFM over existing baselines.

---

引用前请核对原文;本文由星际流动整理:https://feed.astralor.com/items/itm_39ac28e508d01204e4b89145