论文34

多模态智能体自我改进方法跨任务迁移综述

原标题:This survey argues that self-improvement tricks for multimodal agents only carry over to tasks that have what the trick relies on, like c...

Rohan Paul@rohanpaul_ai发布时间 (北京时间)

一篇综述指出,多模态智能体的自我改进技巧只能迁移到具备其所依赖条件(如可校验答案)的任务上,在图像问答中有效的自我学习放到机器人等无可校验信号的任务上可能失效。

中文译文尚未完成,先为你展示原文。

This survey argues that self-improvement tricks for multimodal agents only carry over to tasks that have what the trick relies on, like checkable answers.

AI agent learn from its own attempts in 1 area, like questions about images, can fail in another, like robotics, where there is nothing to check.

Self-improving agents now span visual reasoning, image generation, app and web navigation, and robotics, but each method is tested in only 1 setting. Majority voting, for example, scores short visual answers but finds nothing to compare in robot action trajectories.

The author reviewed methods in the 4 settings and listed each trick's needs, like answers a vote can compare or results a program can check. Each trick then gets a verdict per setting: portable, conditional, blocked, or redundant.

In 1 cited audit of 500 questions, 24% of kept majority-vote labels were wrong, against 5.6% for program-computed answers. These verdicts are predictions, not results from transfer experiments.

Before moving a self-improvement trick to a new task, confirm that task offers the check the trick relies on, then test against a same-budget baseline.

Agent 智能体推理能力多模态论文研究#论文/研究#Agent#多模态#推理