HF Daily Papers 2026-07-10

范围：优先 RLHF / preference optimization / alignment 在 Diffusion 或生成式 Diffusion 中的应用；其次为 RLHF 在 VLM / LLM 中的工作。排除 VLA、机器人、具身智能与 robot manipulation。

筛选说明

• 本列表只基于 Hugging Face Daily Papers 可见元数据与摘要完成初筛和排序；不应将作者摘要中的主张视为已被独立验证的事实。
• 中文 AI Summary 是摘要级判断，不等同于全文结论。
• 相关论文数：1。
• 全文精读候选数：1。

排序后的相关论文

1. UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma

• 方向： RLHF-VLM/LLM
• 研究阅读价值： 4/5
• 潜在工程价值： 4/5
• 作者： Chongyu Fan, Pengfei Liu, Jingjia Huang, Sijia Liu, Yi Lin
• Hugging Face： 论文页
• arXiv： 摘要页 · PDF
• 代码： 未提供
• 项目页： https://chongyu-fan.netlify.app/posts/up/

中文 AI Summary

该论文关注 LLM 强化学习后训练中重要性采样带来的探索—稳定性矛盾：传统 clipping 虽稳定训练，却可能截断低置信度正确推理路径的更新。作者提出 UP，通过对正优势样本使用由 stop-gradient 锚定的非对称、非截断梯度，同时保留负优势样本的 clipping，并声称可迁移到 GRPO、DAPO 与 GSPO 等框架。以上均为基于摘要的判断，实际增益、稳定性边界与适用条件仍需全文验证。

Abstract

Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs). To achieve sample efficiency, modern RL frameworks rely on importance sampling (IS). However, these algorithms suffer from an exploration-stability dilemma. Pure IS often leads to catastrophic training instability, while standard clipping mechanisms used to mitigate this instability strictly constrain the policy update budget. By formalizing the concept of Probability Capacity (Cap), we reveal that conservative clipping structurally stifles exploration by prematurely truncating the update budget for correct but low-confidence reasoning paths. To break free from these constraints, we propose Unbounded Positive Asymmetric Optimization (UP), a universal and plug-and-play objective. UP theoretically restructures the optimization process by anchoring the policy to its current state via the stop-gradient operator. This asymmetric design unleashes unclipped, stable gradients for positive advantages to maximize exploration, while maintaining standard clipping safeguards for negative advantages to prevent training instability. Furthermore, our formulation readily extends across different optimization granularities, including token-level (GRPO, DAPO) and sequence-level (GSPO) frameworks. Extensive experiments demonstrate that UP enhances exploration capacity and achieves superior reasoning accuracy across diverse RL algorithms (DAPO, GSPO, and GRPO), model architectures (Dense, MoE, and vision-language), and training modalities (language and multimodal), validating UP as a truly universal plug-and-play enhancement for RL-based training.

排序依据与全文待核验点

• 为什么相关： Substantively proposes a new RL post-training objective for LLM reasoning, targeting the exploration-versus-stability trade-off in importance-sampled policy optimization; reports applicability across GRPO, DAPO, GSPO, dense/MoE, and vision-language settings.
• 全文待核验： Verify the exact UP objective and ablations showing that its claimed gains come from the asymmetric positive-advantage update rather than training-budget, reward-model, data, or hyperparameter differences.

---

筛选备注

Only 2607.06987 has RL/post-training as a substantive central contribution within the requested scope. The diffusion papers are excluded because their abstracts center on real-time generation, video reasoning, or inference-time search rather than RLHF/preference optimization/reward modeling/alignment/post-training.