NeurIPS 2026 Spotlight
Flow Matching Reinforcement Learning via SDE Inference
When does reinforcement learning with stochastic sampling improve a flow model's deterministic policy?
Abstract
Reinforcement learning is a promising approach for fine-tuning flow matching models. A common strategy uses stochastic SDE inference during training to support exploration, then returns to deterministic ODE inference at deployment. We develop a unified framework based on diffusion-flow duality that covers both reward fine-tuning of generative models and policy optimization with flow policies. We prove that the two inference processes induce close terminal distributions when the flow-matching error is small. We further give sufficient conditions under which policy improvements during stochastic training transfer to deterministic deployment. The analysis explains failure modes caused by weak pretraining or excessive drift during RL. Experiments on synthetic and continuous-control tasks validate the theory and evaluate the resulting algorithms.
Key results
A unified RL framework
Connect reward fine-tuning and flow-policy optimization through stochastic inference, recovering existing methods and motivating new algorithms.
A deployment guarantee
Bound the difference between SDE and ODE terminal distributions and identify when a training-time policy improvement survives deterministic deployment.
Theory meets control
Use synthetic experiments and continuous-control tasks to test the theory and the proposed flow-policy optimization algorithms.
BibTeX
Provisional citation based on the authors' acceptance information. Official BibTeX pending.
% Provisional citation; official BibTeX pending.
@inproceedings{su2026flowsde,
title = {{Flow Matching Reinforcement Learning via SDE Inference}},
author = {Maojiang Su and Tsung-En Lin and Jerry Yao-Chieh Hu and Guo Ye and Haoran Lu and Shang Wu and Zhaoran Wang and Han Liu},
booktitle = {Neural Information Processing Systems (NeurIPS) 2026},
year = {2026}
}