tau2 Bench

116 models evaluated

#	Model	Provider	Type	Score
1	GLM-5V-Turbo	Z.AI	Closed	98.5%
2	GLM-5-Turbo	Z.AI	Closed	98.5%
3	GLM-5	Z.AI	Open	98.2%
4	GLM-5.1	Z.AI	Open	97.7%
5	Qwen3.6 Plus	Alibaba	Closed	97.7%
6	Grok 4.3	xAI	Closed	97.7%
7	DeepSeek V4 Pro (Max)	DeepSeek	Open	96.2%
8	Kimi K2.6	Moonshot AI	Open	95.9%
9	Kimi K2.5 (Reasoning)	Moonshot AI	Closed	95.9%
10	GLM-4.7	Z.AI	Open	95.9%
11	Kimi K2.5	Moonshot AI	Open	95.9%
12	Qwen 3.6 Max (preview)	Alibaba	Closed	95.9%
13	Gemini 3.1 Pro	Google	Closed	95.6%
14	Qwen3.5 397B (Reasoning)	Alibaba	Open	95.6%
15	DeepSeek V4 Flash (High)	DeepSeek	Open	95.6%
16	Gemini 3.5 Flash	Google	Closed	95.3%
17	Qwen3.6-35B-A3B	Alibaba	Open	95.3%
18	DeepSeek V4 Flash (Max)	DeepSeek	Open	95%
19	MiMo-V2-Pro	Xiaomi	Closed	95%
20	Qwen3.7 Max	Alibaba	Closed	94.7%
21	DeepSeek V4 Pro (High)	DeepSeek	Open	94.2%
22	Qwen3.6-27B	Alibaba	Open	94.2%
23	MiMo-V2.5-Pro	Xiaomi	Closed	94.2%
24	Mistral Medium 3.5 128B	Mistral	Open	94.2%
25	GPT-5.5	OpenAI	Closed	93.9%
26	Qwen3.5-27B	Alibaba	Open	93.9%
27	Qwen3.5-122B-A10B	Alibaba	Open	93.6%
28	Grok 4.1 Fast (Reasoning)	xAI	Closed	93.3%
29	Hy3 Preview	Tencent	Open	92.7%
30	GPT-5.2-Codex	OpenAI	Closed	92.1%
31	Claude Opus 4.6 (Adaptive)	Anthropic	Closed	92.1%
32	Muse Spark	Meta	Closed	91.5%
33	MiMo-V2-Omni	Xiaomi	Closed	91.2%
34	Trinity-Large-Preview	Arcee AI	Open	90.1%
35	Trinity-Large-Thinking	Arcee AI	Open	90.1%
36	Claude Opus 4.5 Thinking	Anthropic	Closed	89.5%
37	Qwen3.5-35B-A3B	Alibaba	Open	89.2%
38	Claude Opus 4.7 (Adaptive)	Anthropic	Closed	88.6%
39	LFM2.5-8B-A1B	LiquidAI	Open	88.1%
40	GPT-5.4	OpenAI	Closed	87.1%
41	Gemini 3 Pro	Google	Closed	87.1%
42	GPT-5 (medium)	OpenAI	Closed	86.5%
43	Claude Opus 4.5	Anthropic	Closed	86.3%
44	GPT-5.3 Codex	OpenAI	Closed	86%
45	Ling 2.6 Flash	InclusionAI	Open	86%
46	Claude Opus 4.6	Anthropic	Closed	84.8%
47	GPT-5.2	OpenAI	Closed	84.8%
48	GPT-5 (high)	OpenAI	Closed	84.8%
49	MiniMax M2.7	MiniMax	Open	84.8%
50	Qwen3.5 397B	Alibaba	Open	83.9%
51	MiMo-V2-Flash	Xiaomi	Open	83.9%
52	GPT-5.4 mini	OpenAI	Closed	83.3%
53	GPT-5.1-Codex-Max	OpenAI	Closed	83%
54	GPT-5.1-Codex	OpenAI	Closed	83%
55	GPT-5.1	OpenAI	Closed	81.9%
56	o3	OpenAI	Closed	80.7%
57	Command A+	Cohere	Open	80.7%
58	Claude Sonnet 4.6	Anthropic	Closed	79.5%
59	DeepSeek V3.2	DeepSeek	Open	78.9%
60	GLM-4.6	Z.AI	Open	76.9%
61	GPT-5.4 nano	OpenAI	Closed	76%
62	Grok Code Fast 1	xAI	Closed	75.7%
63	Grok 4	xAI	Closed	74.9%
64	K-Exaone	LG AI Research	Closed	74.3%
65	Qwen3 Max	Alibaba	Closed	74.3%
66	Claude Opus 4.7	Anthropic	Closed	74%
67	Claude 4.1 Opus Thinking	Anthropic	Closed	71.4%
68	GPT-OSS 120B	OpenAI	Open	65.8%
69	Grok 4 Fast (Reasoning)	xAI	Closed	65.8%
70	Grok 4.1 Fast	xAI	Closed	63.7%
71	o1	OpenAI	Closed	62.6%
72	Kimi K2	Moonshot AI	Closed	61.1%
73	GPT-OSS 20B	OpenAI	Open	60.2%
74	Gemma 4 31B	Google	Open	59.9%
75	Gemini 2.5 Pro	Google	Closed	54.1%
76	GPT-4.1 mini	OpenAI	Closed	52.9%
77	Claude 4 Sonnet	Anthropic	Closed	52.3%
78	GPT-4.1	OpenAI	Closed	47.1%
79	Sarvam 105B	Sarvam	Open	46.8%
80	GLM-4.5-Air	Z.AI	Closed	46.5%
81	Nemotron 3 Nano Omni 30B A3B	NVIDIA	Open	45.3%
82	Gemma 4 26B A4B	Google	Open	43.6%
83	Gemini 3 Flash	Google	Closed	43.3%
84	Mistral Small 4 (Reasoning)	Mistral	Open	41.2%
85	Mistral Small 4	Mistral	Open	41.2%
86	DeepSeek V3.1 (Reasoning)	DeepSeek	Open	37.4%
87	DeepSeek-R1	DeepSeek	Open	36.5%
88	DeepSeek V3.1	DeepSeek	Open	34.8%
89	Sarvam 30B	Sarvam	Open	34.5%
90	Solar Pro 2	Upstage	Closed	31.9%
91	Gemini 3.1 Flash-Lite	Google	Closed	31.3%
92	Mistral Large 2	Mistral	Closed	30.7%
93	o3-mini	OpenAI	Closed	28.7%
94	Nemotron 3 Nano 30B	NVIDIA	Open	25.4%
95	GPT-4o	OpenAI	Closed	25.1%
96	Mistral Large 3	Mistral	Closed	24.6%
97	Mistral Medium 3	Mistral	Closed	24.3%
98	DeepSeek V3	DeepSeek	Open	22.8%
99	Granite-4.0-1B	IBM	Open	22.8%
100	Claude 3 Haiku	Anthropic	Closed	21.1%
101	Gemma 4 E4B	Google	Open	20.8%
102	Gemma 4 E2B	Google	Open	20.8%
103	Exaone 4.0 1.2B	LG AI Research	Open	20.5%
104	Granite-4.0-H-1B	IBM	Open	19.6%
105	Llama 3.1 405B	Meta	Open	19%
106	Llama 4 Maverick	Meta	Open	17.8%
107	GPT-4.1 nano	OpenAI	Closed	17.3%
108	Llama 4 Scout	Meta	Open	15.5%
109	Gemini 2.5 Flash	Google	Closed	14.9%
110	Granite-4.0-H-350M	IBM	Open	14.6%
111	Nova Pro	Amazon	Closed	14%
112	Granite-4.0-350M	IBM	Open	13.2%
113	Nemotron Ultra 253B	NVIDIA	Open	11.4%
114	Gemma 3 27B	Google	Open	10.5%
115	Exaone 4.0 32B	LG AI Research	Open	4.1%
116	Phi-4	Microsoft	Open	0%