论文
15AI 评分(两次独立评分的平均)

METR:现实任务构建成本更高

原标题:Separately, we found realistic tasks were harder to make. We quantified this by estimating each task’s “lifecycle cost”: the cost to buil...

METR@METR_Evals发布时间 (北京时间)

另外,我们发现现实任务更难构建。我们通过估算每个任务的“生命周期成本”来量化这一点:构建该任务的成本,以及随后运行并评分100次的成本。与先前工作中的“混乱度”一样,这是我们可以分析的另一个任务差异维度。

中文译文尚未完成,先为你展示原文。

Separately, we found realistic tasks were harder to make. We quantified this by estimating each task’s “lifecycle cost”: the cost to build the task, and then run and score it 100 times. Like “messiness” in previous work, it’s another axis of task variation that we can analyze.

论文研究评测基准#论文/研究#评测/基准