论文
71AI 评分(两次独立评分的平均)

密歇根大学论文发现 AI 智能体在无效应数据中仍报告发现,最高达 98%

原标题:New University of Michigan paper finds that AI agents asked to analyze data where nothing was going on still reported a discovery in up t...

Rohan Paul@rohanpaul_ai发布时间 (北京时间)

密歇根大学一篇新论文发现,让 AI 智能体分析本无任何效应的模拟数据时,最多 98% 的运行仍报告了发现。研究让 4 个前沿智能体在无真实效应的模拟数据中各找一个有说服力的结论,它们报告 p 值小于 0.05 的比例在 54% 到 98% 之间,且全部为假阳性。

中文译文尚未完成,先为你展示原文。

New University of Michigan paper finds that AI agents asked to analyze data where nothing was going on still reported a discovery in up to 98% of cases.

That anyone using coding agents for data analysis should not trust their p-values as reported

And replaying them on subsamples fixes it.

Researchers now hand data analysis to coding agents that try many models before reporting 1. Classic p-values assume the test was chosen before seeing the data.

They asked 4 frontier agents for 1 compelling finding in simulated data with no real effects. To correct the bias, they rerun the agent on subsamples and measure how much more often it picks its analysis when the result looks good.

The agents reported a p-value under 0.05 in 54% to 98% of runs, all false. The correction cut false positives to 5.1% in simulations and 5.9% on data from 3 published papers.

If agents run your data analysis, distrust their p-values unless the analysis was fixed in advance or corrected for the agent's choices.

Agent 智能体数据与训练论文研究#论文/研究#Agent#数据/训练