HF Daily Papers 2026-07-29

范围：优先 RLHF / preference optimization / alignment 在 Diffusion 或生成式 Diffusion 中的应用；其次为 RLHF 在 VLM / LLM 中的工作。排除 VLA、机器人、具身智能、robot manipulation、coding / software-engineering agents、long-horizon agents 与 agentic-RL。

筛选说明

• 本列表只基于 Hugging Face Daily Papers 可见元数据与摘要完成初筛和排序；不应将作者摘要中的主张视为已被独立验证的事实。
• 中文 AI Summary 是摘要级判断，不等同于全文结论。
• 相关论文数：1。
• 全文精读候选数：1。

排序后的相关论文

1. ⭐ Pass the Baton: Trajectory-Relayed On-Policy Distillation

• 方向： RLHF-VLM/LLM
• 研究阅读价值： 4/5
• 潜在工程价值： 4/5
• 作者： Haolei Xu, Xiaowen Xu, Haiwen Hong, Zixuan Ni, Hongxing Li, Yiwen Qiu, Weiming Lu, Yongliang Shen
• Hugging Face： 论文页
• arXiv： 摘要页 · PDF
• 代码： https://github.com/ZJU-REAL/Relay-OPD
• 项目页： https://zju-real.github.io/Relay-OPD/

中文 AI Summary

本文针对大语言模型在线策略蒸馏（OPD）中学生模型在错误前缀上继续偏离导致无效监督的问题，提出了 Relay-OPD 方法。该方法利用教师与学生在失败前缀上的接续不对称性作为无标签手递手触发器，在关键节点由教师接管生成引导片段后再交还给学生继续生成。在 8 个数学推理基准上的实验表明，Relay-OPD 显著提升了 0.6B 与 1.7B 学生的推理准确率，同时将训练轨迹长度减少了 50% 以上。结论基于摘要报道。

Abstract

On-policy distillation (OPD) grounds token-level supervision in the student’s own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks, Relay-OPD achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.

排序依据与全文待核验点

• 为什么相关： Proposes Relay On-Policy Distillation (Relay-OPD) to fix prefix failure in on-policy distillation for LLMs by using label-free handoff triggers where the teacher takes over on failed prefixes.
• 全文待核验： Verify the exact handoff trigger condition math/heuristics, stability under different student-teacher capacity gaps, and actual training compute savings compared to standard OPD.

---

筛选备注

Filtered out non-scoped papers covering VLA/robotics (2607.25895), retrieval/agent search systems (2607.24223, 2607.25431, 2607.24368), UI design decomposition agents (2607.25565), and streaming video tokenizers (2607.24904). Only 2607.26057 falls squarely into the RLHF-VLM/LLM post-training/distillation scope.