Can I contribute FlowRL - a new RL algorithm for LLM reasoning?

[Xuekai-Zhu]

Hi maintainers,

I would like to contribute FlowRL, a new RL algorithm for LLM reasoning that uses distribution matching instead of reward maximization.

Key idea

• Uses distribution matching (via flow balance) rather than reward maximization
• Achieves better generation diversity by avoiding single-peak convergence
• Improves policy generalization
• Potential to handle multiple diverse reward functions in the future

Algorithm

$$ \mathcal{L}_{\text{FlowRL}} = w \cdot \left( \log Z_{\phi}(x) + \frac{1}{|y|} \log \pi_{\theta}(y \mid x) - \beta \hat{r}(x, y) - \frac{1}{|y|} \log \pi_{\text{ref}}(y \mid x) \right)^2 $$

References

• 🤗 HuggingFace Paper : https://huggingface.co/papers/2509.15207
• 🔧 veRL official PR: [recipe] feat: add FlowRL recipe volcengine/verl#3924
• 💻 Source code: https://github.com/Xuekai-Zhu/FlowRL

Would this be a good fit for this repository? Happy to discuss implementation details!

Thanks!