论文38
LlamaIndex 推出 ParseBench-2 表单解析基准
原标题:Parsing forms is a hard problem, whether with specialized OCR or frontier models.
LlamaIndex 在 ParseBench-2 中构建了表单解析数据集,含 30 个国家 244 个政府表单模板的 600 页、2 万个标注字段(文本、复选框、签名)。该基准还设计了按严重程度加权的表单指标:空白字段须保持空白、格式变化不计分、丢失签名计 5、数字错误至少计 10。数据集已在 Hugging Face 发布,详情见博客与技术报告。
中文译文尚未完成,先为你展示原文。
Parsing forms is a hard problem, whether with specialized OCR or frontier models.
There is no standardization across various form templates. There are multiple form elements (e.g. checkboxes, signatures, text fields) that need to be represented in various ways. A good parser needs to both extract empty/filled form values along with exact bounding boxes.
We’ve spent time carefully crafting a forms dataset within ParseBench-2. It consists of 600 pages from 244 government form templates across 30 countries, with 20K annotated fields (text, checkboxes, signatures).
We also designed a forms metric that’s weighted by severity policies, like the following:
- Blank fields have to stay blank
- Formatting changes cost nothing
- A lost signature counts 5
- A wrong number costs min 10
Different form extraction use cases will prioritize different errors more, and we encourage you to tune your own severity policies for form parsing.
More details in our blog / tech report: https://www.llamaindex.ai/blog/introducing-parsebench-v2
Benchmark: https://www.parsebench.ai/
Available on Hugging Face! https://huggingface.co/datasets/llamaindex/ParseBench-2
