论文58

TokenWave AI 发布 SWE-CoWork 基准,测试编码智能体能否接手队友未完成的工作

原标题:Coding-agent benchmarks are starting to look like a new hire’s first week.

Rohan Paul@rohanpaul_ai发布时间 (北京时间)

TokenWave AI 发布 SWE-CoWork 基准,用于测试编码智能体能否像真实团队成员一样接手队友未完成的工作。该基准覆盖 101 个真实开源仓库的 101 项任务、26 个模拟队友、11 种编程语言、12 个应用领域和 17 种职场工具,82% 的需求掌握在队友手中,智能体需主动询问才能确定要构建什么。

中文译文尚未完成,先为你展示原文。

Coding-agent benchmarks are starting to look like a new hire’s first week.

TokenWave AI has released SWE-CoWork, a new benchmark that tests whether AI coding agents can work like real team members.

Giving GPT-6 Astra the full spec up front lifted its passed checks by 16 points, yet it still fully finished only 8 of 101 tasks. SWE-CoWork puts coding agents in a simulated workplace modeled on real-world workflows, then links each failed check to a specific requirement.

Most coding benchmarks hand an AI agent a complete written task, but SWE-CoWork puts it in a simulated workplace where 82% of the requirements sit with teammates, so the agent only learns what to build by asking the right person. The best models pass about 63% of the individual checks yet fully finish fewer than 8% of 101 tasks.

Can a coding agent take over a teammate’s work in progress? Introducing SWE-CoWork, a benchmark for coding agents working as digital employees on long-horizon software engineering tasks. Each assignment starts with a short handoff in a simulated workplace modeled on real-world engineering workflows. Agents search task boards and versioned docs, ask teammates for missing details, and revise their code when a reply changes what needs to be built. Open questions and confirmed requirements can both be lost before the agent hands the work back. SWE-CoWork lets us trace them through the conversation, the patch, and the tests. 101 assignments across 101 real open-source repositories. 26 simulated teammates per workplace. 11 programming languages. 12 application domains. 17 workplace tools. 🌐 Website: https://tokenwave-ai.github.io/swe-cowork-page/ 📄 Paper: https://arxiv.org/abs/2611.20218 💻 Code: https://github.com/TokenWave-AI/swe-cowork-bench 🤗 Hugging Face Daily Papers: https://huggingface.co/papers/2611.20218 (If you find it useful, we’d appreciate an upvote!)
查看原帖
Agent 智能体AI 编码论文研究评测基准#论文/研究#Agent#编码#评测/基准

同一件事的其他报道1 篇