---
title: "Sequentially-Controlled Interactive Multi-Particle Flow-Maps for Online Feedback-Driven Search"
source: "arXiv AI/CL/LG（论文）"
source_tier: "t2"
url: "https://arxiv.org/abs/2607.01144v1"
permalink: "https://feed.astralor.com/items/itm_39ac28e508d01204e4b89145"
published: "2026-07-01T16:27:17.000Z"
score: 59
channel: "ai"
tags: ["IMPFM","Online Learning","multi-particle system","exploration-exploitation tradeoff","flow mapping"]
---

# Sequentially-Controlled Interactive Multi-Particle Flow-Maps for Online Feedback-Driven Search

## At a glance

This paper proposes the IMPFM framework, which uses multi-particle interactive flow-maps for sample-efficient online feedback-driven search, addressing broad exploration when preferences are unknown a priori. It features a posterior sample sharing mechanism and an exploration-exploitation reweighting strategy, with theoretical proof that it steers the multi-particle system toward a KL-tilted target distribution, preventing mode collapse. Extensive empirical evaluations show IMPFM significantly outperforms existing baselines across diverse search and alignment tasks.

## Key points

- Proposes IMPFM, combining flow-maps with sequentially-controlled multi-particle dynamics for global exploration and feedback-driven alignment
- Introduces a novel posterior sample sharing mechanism that corrects individual drift using collective posterior samples, maximizing sample utility
- Theoretically proves that a multi-particle interaction-aware Feynman-Kac corrector progressively steers the system toward a KL-tilted distribution, preserving diversity

## Full text

While generative models have enabled training-free reward alignment, current methods typically excel in local exploration within narrow regions of the underlying distribution. These approaches struggle when preferences are unknown a priori and only revealed through sequential feedback-a scenario demanding broad exploration to uncover high-utility regions. To address this, we propose Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for sample-efficient online feedback-driven search. IMPFM progressively transports a group of interactive particles toward the target distribution, maintaining the broad coverage essential for heterogeneous preference alignment. IMPFM introduces a principled and efficient posterior sample sharing mechanism across particles powered by flow maps. By correcting individual particle drift with the collective posterior samples of the entire ensemble at each resampling step, the framework maximizes sample utility to enable global exploration while actively mitigating reward over-optimization, typical of standard control frameworks. Paired with a principled exploration-exploitation reweighting mechanism involving multi-particle interaction, this sequentially corrected multi-particle dynamics explicitly preserves structural diversity and overcomes the weight degeneracy inherent to standard SMC samplers. Crucially, we prove that the resulting sampling framework yields a multi-particle interaction-aware Feynman-Kac corrector that progressively steers the multi-particle system toward a KL-tilted target distribution, facilitating global exploration and preventing mode collapse. Extensive empirical evaluations and rigorous ablations across diverse search and alignment tasks confirm the efficacy of IMPFM over existing baselines.

---

Verify against the original before citing; curated by Astral Feed: https://feed.astralor.com/items/itm_39ac28e508d01204e4b89145