论文64

微软论文:LLM 遭错误反驳时会放弃正确答案,准确率训练无法修复

原标题:New Microsoft paper finds that LLMs give up correct answers when challenged with wrong arguments, and accuracy training does not fix it.

Rohan Paul@rohanpaul_ai发布时间 (北京时间)

微软新论文发现,当用户或其他智能体用错误论据质疑答案时,LLM 常常放弃原本正确的答案,增加准确率训练、采样或自我反思都无法解决。论文提出 Self-Disagreement 方法,让同一个模型先给出答案,再写出最强反驳理由,然后对该理由作出裁决并重新作答,标准答案检查器同时奖励最终答案和裁决。

中文译文尚未完成,先为你展示原文。

New Microsoft paper finds that LLMs give up correct answers when challenged with wrong arguments, and accuracy training does not fix it.

When a user or another agent disputes an answer, models often abandon correct answers even when the challenge is wrong. More accuracy training, more sampling, or self-reflection does not fix it.

Self-Disagreement trains one model to state an answer, write the strongest case against it, then rule on that case and answer again. A standard answer checker rewards the final answer and the ruling.

When challengers were wrong, base Qwen3-8B kept its correct answer 53.3% of the time and Gemma-2 9B 25.1%. After training, both kept it about 89% of the time, while accuracy-only training added no such skill.

If your model faces pushback from users or other agents, test how often it drops correct answers, since accuracy scores will not show it.

Microsoft / Copilot数据与训练安全对齐论文研究#论文/研究#安全/对齐#数据/训练#Microsoft