产品58

Perplexity 推出 Orchestrator API,主打 KV cache 友好的决策模型路由

原标题:KV-cache compatible decision-model based routing on Perplexity Orchestrator API

Aravind Srinivas@AravSrinivas发布时间 (北京时间)

Perplexity 推出 Orchestrator API,在一个 API 中接入 45+ 开源权重与闭源模型,支持自动供应商故障切换和统一计费。其路由设计对 KV cache 友好,由用户决定哪些任务使用哪些模型,而非通用自动路由。示例中把 fable 5.2 拒绝的安全与生物任务转给 gpt-6.1 sol,在 terminal-bench 4.1 上从 63 题解出 38 题提升到 41 题。

中文译文尚未完成,先为你展示原文。

KV-cache compatible decision-model based routing on Perplexity Orchestrator API

we are launching the Orchestrator API: 45+ open-weight and proprietary models in one api, with automatic provider failover and unified billing. general model routing doesn't really work in practice: it breaks the kv cache and is usually overfit to a few benchmarks. so we designed our routing api differently: it's kv cache friendly, and you decide which models to use for which tasks. one example: fable 5.2 often refuses benign security and biology tasks, so we route those (and anything after a refusal) to gpt-6.1 sol. on terminal-bench 4.1 this goes from 38 to 41 solved tasks out of 63.
查看原帖
OpenAI / ChatGPTAgent 智能体部署工程产品更新#产品更新#Agent#部署/工程#OpenAI

同一件事的其他报道5 篇