跳到正文
arXiv Artificial Intelligence· Hamed Mahdavi (Pennsylvania State University), Pouria Mahdavinia (Pennsylvania State University), Majid Farhadi (Google), Vasant Honavar (Pennsylvania State University)·· 20 小时前AI 评分32

SPLICE:将提示拼接进推理轨迹,从演示者失败中学习

Splicing Hints into Reasoning Traces: Learning from Demonstrator Failures

AI 导读

SPLICE 将验证与修复移出训练循环,改为离线数据生成:第二个模型仅凭参考答案,在推理模型生成轨迹时验证进展、在最后已验证步骤处截断失败轨迹,并拼接自适应增强的提示直至进展恢复。用 SPLICE 轨迹微调的模型在 AIME 上超过用未编辑轨迹蒸馏的同款模型,并在经人工评分验证的评分协议下于奥赛证明题上同样领先。训练后模型的轨迹在难点处会以拼接提示的风格宣告分支并采取更高风险步骤,预测熵随之上升。

来源:arXiv Artificial Intelligence · arxiv.org