论文
28AI 评分(两次独立评分的平均)

METR 提出模型能力新指标

原标题:We fit a metric of model ability on our tasks as an alternative measure of model ability. Like Epoch's ECI, this estimates task difficult...

METR@METR_Evals发布时间 (北京时间)

我们在自己的任务上拟合了一个模型能力指标,作为模型能力的替代衡量方式。与 Epoch 的 ECI 类似,该指标直接从智能体在各任务上的得分来估计任务难度和智能体能力,因此不需要人工时间估算。 https://x.com/EpochAIResearch/status/1982888284436218275?s=20

中文译文尚未完成,先为你展示原文。

We fit a metric of model ability on our tasks as an alternative measure of model ability. Like Epoch's ECI, this estimates task difficulties and agent abilities directly from agent scores across tasks, so it doesn't need human time estimates.
https://x.com/EpochAIResearch/status/1982888284436218275?s=20

We've launched a new tool to track AI progress! The tool addresses one of the field's biggest challenges: benchmark saturation. It's called the Epoch Capabilities Index (ECI) — here's what makes it different:
查看原帖
论文研究评测基准#论文/研究#评测/基准

同一件事的其他报道1 篇