观点
22AI 评分(两次独立评分的平均)

Jerry Liu:构建优质评测基准是企业 ROI 的核心技能

原标题:An extension of this point is that building proper evals/benchmarks is a universally useful skill, and the core necessary ingredient for ...

Jerry Liu@jerryjliu0发布时间 (北京时间)

Jerry Liu 认为,构建合适的评测与基准是普遍有用的技能,也是企业获得 ROI 的核心要素。最难的是定义任务中"好"的标准并衡量它,之后实现并不难;许多公开基准在任务设置、环境、指标和标注上存在可改进的缺陷。他表示,为社区做贡献的最佳方式是完全公开地构建基准,围绕文档解析与文档工作打造一套开放工具,供任何人分析和按需调整。

中文译文尚未完成,先为你展示原文。

An extension of this point is that building proper evals/benchmarks is a universally useful skill, and the core necessary ingredient for enterprise ROI.

The hardest part is defining what 'good' looks like for any task and how to measure it. Afterwards, the implementation is easy.

Building good benchmarks is not easy, and many public benchmarks have imperfections in the task setup, environment, metrics, and annotations that could be revised.

The best way for us to contribute to the broader community here is to build benchmarks completely in the open. We want to build an open set of tools around document parsing and document work that anyone can analyze and adapt to their needs.

评测基准大佬观点#大佬观点#评测/基准