Benchmarks and Evals

My team builds benchmarks and environments for coding agents, tool and MCP use, computer use, and expert-level professional work, and develops the evaluation methodology behind Scale’s public leaderboards.

Recent papers include READY (Chatrath et al., 2026), a framework for qualifying agents for reliable enterprise deployment under human oversight and cost constraints; ChainWorld (Siu et al., 2026), which composes atomic OSWorld tasks into long-horizon desktop workloads and shows that current computer-use agents complete at most 31% of them; and DrugDiscoveryBench (Akyürek et al., 2026), an expert-curated benchmark of early-stage drug-discovery tasks for coding agents (see also AI for Science).

Publications

2026

arXiv

BARC-Bench: Benign Asks, Risky Consequences

Kangkang Li, Kelvin Luu, Lily Ma, Tu Trinh, Ying Liu, and Mohamed Elfeki

arXiv preprint arXiv:2610.32024, Oct 2026

arXiv Bib

@article{li2026barcbench,
  title = {{BARC}-Bench: Benign Asks, Risky Consequences},
  author = {Li, Kangkang and Luu, Kelvin and Ma, Lily and Trinh, Tu and Liu, Ying and Elfeki, Mohamed},
  journal = {arXiv preprint arXiv:2610.32024},
  year = {2026},
  month = oct,
}

arXiv

Maintaining Benchmarks Against Increasingly Capable Agents: Detection and Remediation of Unearned Passes

Weijun Luo, Kelvin Luu, Xinyi Liu, Guangze Luo, Miguel Romero Calvo, Soham Dan, Daniel Yue Zhang, Ying Liu, and Mohamed Elfeki

arXiv preprint arXiv:2609.34262, Sep 2026

arXiv Bib

@article{luo2026maintaining,
  title = {Maintaining Benchmarks Against Increasingly Capable Agents: Detection and Remediation of Unearned Passes},
  author = {Luo, Weijun and Luu, Kelvin and Liu, Xinyi and Luo, Guangze and Romero Calvo, Miguel and Dan, Soham and Zhang, Daniel Yue and Liu, Ying and Elfeki, Mohamed},
  journal = {arXiv preprint arXiv:2609.34262},
  year = {2026},
  month = sep,
}

arXiv
READY or Not: Reliable Enterprise Agent Deployment

Veronica Chatrath, Bryan Zhu, Jingxuan Fan, George Pu, Soham Dinesh Tiwari, Soham Dan, Ryan Young, Yuan Li, Yuang Yao, Apaar Shanker, Minglai Yang, Daniel Yue Zhang, Yunzhong He, Ying Liu, Chenguang Wang, and 2 more authors

arXiv preprint arXiv:2609.02095, Sep 2026

arXiv Bib
@article{chatrath2026ready, title = {{READY} or Not: Reliable Enterprise Agent Deployment}, author = {Chatrath, Veronica and Zhu, Bryan and Fan, Jingxuan and Pu, George and Tiwari, Soham Dinesh and Dan, Soham and Young, Ryan and Li, Yuan and Yao, Yuang and Shanker, Apaar and Yang, Minglai and Zhang, Daniel Yue and He, Yunzhong and Liu, Ying and Wang, Chenguang and Yin, Zhijun and Xue, Yuan}, journal = {arXiv preprint arXiv:2609.02095}, year = {2026}, month = sep, }

Report

DrugDiscoveryBench: Can Coding Agents Assist Early-Stage Drug Discovery?

Afra Feyza Akyürek, Xinming Tu, Alec Gutmanstein, Jason Qin, Divyansh Agarwal, Sofia Monasdotter, Sergey Chekhov, Brenda Hernandez Villegas, Kirill Chugunov, Judah Engel, Veronica Chatrath, Oscar Kavanagh, Geobio Boo, Ernesto Hernandez, Ying Liu, and 7 more authors

Scale AI technical report, with Phylo , Jun 2026

Bib HTML PDF

@techreport{akyurek2026drugdiscoverybench,
  title = {{DrugDiscoveryBench}: Can Coding Agents Assist Early-Stage Drug Discovery?},
  author = {Aky{\"u}rek, Afra Feyza and Tu, Xinming and Gutmanstein, Alec and Qin, Jason and Agarwal, Divyansh and Monasdotter, Sofia and Chekhov, Sergey and Hernandez Villegas, Brenda and Chugunov, Kirill and Engel, Judah and Chatrath, Veronica and Kavanagh, Oscar and Boo, Geobio and Hernandez, Ernesto and Liu, Ying and Xue, Yuan and Sabharwal, Aakash and Zhang, Daniel Yue and Doctor, Zainab and Qu, Yuanhao and He, Yunzhong and Hassaan, Sami},
  institution = {Scale AI (with Phylo)},
  year = {2026},
  month = jun,
}

arXiv

ChainWorld: Composing Long-Horizon Desktop Workloads from Atomic OSWorld Tasks

Vincent Siu, Manasi Sharma, Dawn Song, Daniel Yue Zhang, Chenguang Wang, and Ying Liu

arXiv preprint arXiv:2606.21654, Jun 2026

arXiv Bib

@article{siu2026chainworld,
  title = {{ChainWorld}: Composing Long-Horizon Desktop Workloads from Atomic {OSWorld} Tasks},
  author = {Siu, Vincent and Sharma, Manasi and Song, Dawn and Zhang, Daniel Yue and Wang, Chenguang and Liu, Ying},
  journal = {arXiv preprint arXiv:2606.21654},
  year = {2026},
  month = jun,
}