Dataset Registry

tb datasets list

terminal-bench-core

head

The latest pre-release version of terminal-bench core. Hand crafted by undergraduate, graduate, and industry researchers.

235 tasks

terminal-bench-core

0.1.0

The terminal-bench task set used in our launch. Hand crafted by undergraduate, graduate, and industry researchers.

80 tasks

terminal-bench-core

0.1.1leaderboard

The terminal-bench task set used in our launch. Hand crafted by undergraduate, graduate, and industry researchers. Recently patched to support terminal-bench 0.2.4+.

80 tasks

deveval

head

Adapter for DevEval(https://github.com/open-compass/DevEval). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/1

63 tasks

evoeval-difficult

head

Adapter for EvoEval(https://github.com/evo-eval/evoeval). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/5

100 tasks

appworld-dev

headencrypted

Adapter for AppWorld (https://github.com/StonyBrookNLP/appworld). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/3

57 tasks

swebench-verified

head

Adapter for SWEBench (https://github.com/SWE-bench/SWE-bench). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/2

500 tasks

mlebench-lite

head

Adapter for MLEBench (https://github.com/openai/mle-bench). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/7

20 tasks

sweperf

head

Adapter for SWE-Perf (https://github.com/SWE-Perf/SWE-Perf). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/10

140 tasks

aider-polyglot

head

Adapter for Aider Polyglot (https://github.com/Aider-AI/polyglot-benchmark). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/9

225 tasks

usaco

head

Adapter for USACO (https://github.com/princeton-nlp/USACO). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/11

304 tasks

algotune

head

Adapter for Algotune (https://github.com/oripress/AlgoTune). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/18

154 tasks

appworld-test_challenge

headencrypted

Adapter for AppWorld (https://github.com/StonyBrookNLP/appworld). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/6

appworld-test_normal

headencrypted

Adapter for AppWorld (https://github.com/StonyBrookNLP/appworld). Dataset PR: https://github.com/laude-institute/terminal-bench-datasets/pull/6

168 tasks

Learn more about the Terminal-Bench registry here.