论文58

Epoch AI 用自家工作任务测试 GPT-6.2 Astra 等模型,均未达到员工标准

原标题:Can the most capable LLMs match humans at real work? Not yet, at least not on the kind of work we do at Epoch.

Epoch AI@EpochAIResearch发布时间 (北京时间)

Epoch AI 用其 Epoch Automation Reports 基准测试 GPT-6.2 Astra 等近期模型,任务涵盖制作 Epoch 风格图表到设计基础研究实验,由 Epoch 员工按内部质量标准打分,100% 代表 Epoch 员工应有的水平。

中文译文尚未完成,先为你展示原文。

Can the most capable LLMs match humans at real work? Not yet, at least not on the kind of work we do at Epoch.

We tested GPT-6.2 Astra and other recent models on Epoch Automation Reports, our benchmark of open-ended Epoch work tasks, ranging from making Epoch-style diagrams to designing basic research experiments.

An Epoch employee scores the models’ outputs based on our internal quality standards. A score of 100% represents the standard expected of an Epoch employee.

All models failed to meet the Epoch employee standard. GPT-6.2 Astra ranks fourth (averaging 65%), despite sharing the top ECI score of 174 with Fable 5.2.

Fable 5.2 scored highest, averaging 82%.

On one task, both Fable 5.2 and Opus 5.5 matched the level of a contractor. More on that below.

OpenAI / ChatGPTAgent 智能体评测基准#评测/基准#Agent#OpenAI#Anthropic

同一件事的其他报道1 篇