terminal Bench Hard

115 models evaluated

#	Model	Provider	Type	Score
1	GPT-5.5	OpenAI	Closed	60.6%
2	GPT-5.4	OpenAI	Closed	57.6%
3	Claude Opus 4.7	Anthropic	Closed	54.5%
4	Gemini 3.1 Pro	Google	Closed	53.8%
5	GPT-5.3 Codex	OpenAI	Closed	53.0%
6	GPT-5.4 mini	OpenAI	Closed	52.3%
7	Claude Opus 4.7 (Adaptive)	Anthropic	Closed	51.5%
8	Qwen3.7 Max	Alibaba	Closed	50.8%
9	Claude Opus 4.6	Anthropic	Closed	48.5%
10	GPT-5.2	OpenAI	Closed	47.0%
11	Claude Opus 4.5 Thinking	Anthropic	Closed	47.0%
12	DeepSeek V4 Pro (Max)	DeepSeek	Open	46.2%
13	Claude Sonnet 4.6	Anthropic	Closed	46.2%
14	Claude Opus 4.6 (Adaptive)	Anthropic	Closed	46.2%
15	GPT-5.1	OpenAI	Closed	45.5%
16	Muse Spark	Meta	Closed	45.5%
17	Kimi K2.6	Moonshot AI	Open	43.9%
18	Qwen3.6 Plus	Alibaba	Closed	43.9%
19	Qwen 3.6 Max (preview)	Alibaba	Closed	43.9%
20	GLM-5.1	Z.AI	Open	43.2%
21	GLM-5	Z.AI	Open	43.2%
22	MiMo-V2.5-Pro	Xiaomi	Closed	43.2%
23	GPT-5.4 nano	OpenAI	Closed	42.4%
24	DeepSeek V4 Pro (High)	DeepSeek	Open	41.7%
25	Gemini 3 Pro	Google	Closed	41.7%
26	Gemini 3.5 Flash	Google	Closed	40.9%
27	Qwen3.5 397B (Reasoning)	Alibaba	Open	40.9%
28	Claude Opus 4.5	Anthropic	Closed	40.9%
29	MiMo-V2-Pro	Xiaomi	Closed	40.9%
30	MiniMax M2.7	MiniMax	Open	39.4%
31	DeepSeek V4 Flash (High)	DeepSeek	Open	38.6%
32	GPT-5 (medium)	OpenAI	Closed	37.9%
33	Grok 4	xAI	Closed	37.9%
34	Grok 4.3	xAI	Closed	37.9%
35	GPT-5.2-Codex	OpenAI	Closed	37.1%
36	o3	OpenAI	Closed	37.1%
37	Gemma 4 31B	Google	Open	36.4%
38	DeepSeek V4 Flash (Max)	DeepSeek	Open	35.6%
39	Qwen3.5 397B	Alibaba	Open	35.6%
40	Kimi K2.5 (Reasoning)	Moonshot AI	Closed	34.8%
41	GPT-5.1-Codex-Max	OpenAI	Closed	34.8%
42	Qwen3.6-27B	Alibaba	Open	34.8%
43	Qwen3.6-35B-A3B	Alibaba	Open	34.8%
44	Kimi K2.5	Moonshot AI	Open	34.8%
45	MiMo-V2-Omni	Xiaomi	Closed	34.8%
46	GPT-5.1-Codex	OpenAI	Closed	34.8%
47	Claude 4.1 Opus Thinking	Anthropic	Closed	34.3%
48	Hy3 Preview	Tencent	Open	34.1%
49	Mistral Medium 3.5 128B	Mistral	Open	33.3%
50	GLM-5-Turbo	Z.AI	Closed	33.3%
51	GPT-5 (high)	OpenAI	Closed	32.6%
52	Qwen3.5-27B	Alibaba	Open	32.6%
53	DeepSeek V3.2	DeepSeek	Open	32.6%
54	GLM-5V-Turbo	Z.AI	Closed	32.6%
55	GLM-4.7	Z.AI	Open	31.8%
56	Gemini 3 Flash	Google	Closed	31.8%
57	Qwen3.5-122B-A10B	Alibaba	Open	31.1%
58	GLM-4.6	Z.AI	Open	28.8%
59	Claude 4 Sonnet	Anthropic	Closed	27.3%
60	Gemini 2.5 Pro	Google	Closed	26.5%
61	Qwen3.5-35B-A3B	Alibaba	Open	26.5%
62	MiMo-V2-Flash	Xiaomi	Open	25.8%
63	DeepSeek V3.1 (Reasoning)	DeepSeek	Open	25.0%
64	Command A+	Cohere	Open	25.0%
65	Gemini 3.1 Flash-Lite	Google	Closed	24.2%
66	DeepSeek V3.1	DeepSeek	Open	24.2%
67	Grok 4.1 Fast (Reasoning)	xAI	Closed	24.2%
68	GPT-OSS 120B	OpenAI	Open	23.5%
69	K-Exaone	LG AI Research	Closed	22.7%
70	Trinity-Large-Preview	Arcee AI	Open	22.7%
71	Trinity-Large-Thinking	Arcee AI	Open	22.7%
72	Ling 2.6 Flash	InclusionAI	Open	21.2%
73	GLM-4.5-Air	Z.AI	Closed	20.5%
74	Qwen3 Max	Alibaba	Closed	20.5%
75	Grok 4 Fast (Reasoning)	xAI	Closed	18.9%
76	Grok Code Fast 1	xAI	Closed	17.4%
77	Mistral Small 4 (Reasoning)	Mistral	Open	17.4%
78	Mistral Small 4	Mistral	Open	17.4%
79	Mistral Large 3	Mistral	Closed	15.9%
80	Kimi K2	Moonshot AI	Closed	15.9%
81	DeepSeek-R1	DeepSeek	Open	15.9%
82	Grok 4.1 Fast	xAI	Closed	14.4%
83	GPT-4.1	OpenAI	Closed	13.6%
84	Gemma 4 26B A4B	Google	Open	13.6%
85	o1	OpenAI	Closed	12.9%
86	Gemini 2.5 Flash	Google	Closed	12.1%
87	Nemotron 3 Nano 30B	NVIDIA	Open	12.1%
88	GPT-OSS 20B	OpenAI	Open	10.6%
89	GPT-4o	OpenAI	Closed	8.3%
90	Nemotron 3 Nano Omni 30B A3B	NVIDIA	Open	8.3%
91	Gemma 4 E4B	Google	Open	8.3%
92	GPT-4.1 mini	OpenAI	Closed	7.6%
93	o3-mini	OpenAI	Closed	6.8%
94	Llama 3.1 405B	Meta	Open	6.8%
95	DeepSeek V3	DeepSeek	Open	6.8%
96	Llama 4 Maverick	Meta	Open	6.8%
97	Mistral Large 2	Mistral	Closed	6.1%
98	Nova Pro	Amazon	Closed	6.1%
99	Solar Pro 2	Upstage	Closed	4.5%
100	Phi-4	Microsoft	Open	3.8%
101	GPT-4.1 nano	OpenAI	Closed	3.8%
102	Gemma 3 27B	Google	Open	3.8%
103	Mistral Medium 3	Mistral	Closed	3.8%
104	Gemma 4 E2B	Google	Open	3.0%
105	Nemotron Ultra 253B	NVIDIA	Open	2.3%
106	Sarvam 30B	Sarvam	Open	2.3%
107	Sarvam 105B	Sarvam	Open	1.5%
108	Llama 4 Scout	Meta	Open	1.5%
109	Exaone 4.0 32B	LG AI Research	Open	1.5%
110	Claude 3 Haiku	Anthropic	Closed	0.8%
111	Granite-4.0-1B	IBM	Open	0.0%
112	Granite-4.0-H-1B	IBM	Open	0.0%
113	Exaone 4.0 1.2B	LG AI Research	Open	0.0%
114	Granite-4.0-350M	IBM	Open	0.0%
115	Granite-4.0-H-350M	IBM	Open	0.0%