论文58
Epoch AI 用自家工作任务测试 GPT-6.2 Astra 等模型,均未达到员工标准
原标题:Can the most capable LLMs match humans at real work? Not yet, at least not on the kind of work we do at Epoch.
Epoch AI 用其 Epoch Automation Reports 基准测试 GPT-6.2 Astra 等近期模型,任务涵盖制作 Epoch 风格图表到设计基础研究实验,由 Epoch 员工按内部质量标准打分,100% 代表 Epoch 员工应有的水平。
中文译文尚未完成,先为你展示原文。
Can the most capable LLMs match humans at real work? Not yet, at least not on the kind of work we do at Epoch.
We tested GPT-6.2 Astra and other recent models on Epoch Automation Reports, our benchmark of open-ended Epoch work tasks, ranging from making Epoch-style diagrams to designing basic research experiments.
An Epoch employee scores the models’ outputs based on our internal quality standards. A score of 100% represents the standard expected of an Epoch employee.
All models failed to meet the Epoch employee standard. GPT-6.2 Astra ranks fourth (averaging 65%), despite sharing the top ECI score of 174 with Fable 5.2.
Fable 5.2 scored highest, averaging 82%.
On one task, both Fable 5.2 and Opus 5.5 matched the level of a contractor. More on that below.
同一件事的其他报道1 篇
Epoch 用自动化报告基准测试 GPT-6.2 Astra
Epoch AI · 原文
