tau2 Bench

142 models evaluated

#	Model	Provider	Type	Score
4	Step 3.7 Flash	StepFun · Open weight	Artificial Analysis τ²-bench	98.5%
5	GLM-5-Turbo	Z.AI · Closed	Artificial Analysis τ²-bench	98.5%
6	GLM-5V-Turbo	Z.AI · Closed	Artificial Analysis τ²-bench	98.5%
7	GLM-5	Z.AI · Open weight	Artificial Analysis τ²-bench	98.2%
8	GLM-5.1	Z.AI · Open weight	Artificial Analysis τ²-bench	97.7%
9	Qwen3.6 Plus	Alibaba · Closed	Artificial Analysis τ²-bench	97.7%
10	Grok 4.3	xAI · Closed	Artificial Analysis τ²-bench	97.7%
11	DeepSeek V4 Pro (Max)	DeepSeek · Open weight	Artificial Analysis τ²-bench	96.2%
12	Kimi K2.6	Moonshot AI · Open weight	Artificial Analysis τ²-bench	95.9%
13	Kimi K2.5	Moonshot AI · Open weight	Artificial Analysis τ²-bench	95.9%
14	GLM-4.7	Z.AI · Open weight	Artificial Analysis τ²-bench	95.9%
15	Qwen 3.6 Max (preview)	Alibaba · Closed	Artificial Analysis τ²-bench	95.9%
16	Kimi K2.5 (Reasoning)	Moonshot AI · Closed	Artificial Analysis τ²-bench	95.9%
17	Gemini 3.1 Pro	Google · Closed	τ²-bench published setup	95.6%
18	Qwen3.5 397B	Alibaba · Open weight	Artificial Analysis τ²-bench	95.6%
19	DeepSeek V4 Flash (High)	DeepSeek · Open weight	Artificial Analysis τ²-bench	95.6%
20	Qwen3.5 397B (Reasoning)	Alibaba · Open weight	Artificial Analysis τ²-bench	95.6%
21	Gemini 3.5 Flash	Google · Closed	Artificial Analysis τ²-bench	95.3%
22	Qwen3.6-35B-A3B	Alibaba · Open weight	Artificial Analysis τ²-bench	95.3%
23	MiniMax M2.5	MiniMax · Closed	Artificial Analysis τ²-bench	95.3%
24	DeepSeek V4 Flash (Max)	DeepSeek · Open weight	Artificial Analysis τ²-bench	95%
25	MiMo-V2-Pro	Xiaomi · Closed	Artificial Analysis τ²-bench	95%
26	Qwen3.7 Max	Alibaba · Closed	Artificial Analysis τ²-bench	94.7%
27	Claude Opus 4.8	Anthropic · Closed	Artificial Analysis τ²-bench	94.4%
28	DeepSeek V4 Pro (High)	DeepSeek · Open weight	Artificial Analysis τ²-bench	94.2%
29	Qwen3.6-27B	Alibaba · Open weight	Artificial Analysis τ²-bench	94.2%
30	Mistral Medium 3.5 128B	Mistral · Open weight	Artificial Analysis τ²-bench	94.2%
31	MiMo-V2.5-Pro	Xiaomi · Closed	Artificial Analysis τ²-bench	94.2%
32	GPT-5.5	OpenAI · Closed	τ²-bench Telecom	93.9%
33	Qwen3.5-27B	Alibaba · Open weight	Artificial Analysis τ²-bench	93.9%
34	Qwen3.5-122B-A10B	Alibaba · Open weight	Artificial Analysis τ²-bench	93.6%
35	Grok 4.1 Fast (Reasoning)	xAI · Closed	Artificial Analysis τ²-bench	93.3%
36	Qwen3.7 Plus	Alibaba · Closed	Artificial Analysis τ²-bench	93%
37	Claude Opus 4.6 (Adaptive)	Anthropic · Closed	Artificial Analysis τ²-bench	92.1%
38	GPT-5.2-Codex	OpenAI · Closed	Artificial Analysis τ²-bench	92.1%
39	Muse Spark	Meta · Closed	τ²-bench published setup	91.5%
40	MiMo-V2-Omni	Xiaomi · Closed	Artificial Analysis τ²-bench	91.2%
41	Trinity-Large-Preview	Arcee AI · Open weight	Artificial Analysis τ²-bench	90.1%
42	Trinity-Large-Thinking	Arcee AI · Open weight	Artificial Analysis τ²-bench	90.1%
43	Kimi K2.7 Code	Moonshot AI · Open weight	Artificial Analysis τ²-bench	90.1%
44	Claude Opus 4.5 Thinking	Anthropic · Closed	Artificial Analysis τ²-bench	89.5%
45	Qwen3.5-35B-A3B	Alibaba · Open weight	Artificial Analysis τ²-bench	89.2%
46	MiniMax M3	MiniMax · Open weight	Artificial Analysis τ²-bench	88.9%
47	Claude Opus 4.7 (Adaptive)	Anthropic · Closed	Artificial Analysis τ²-bench	88.6%
48	Step 3.5 Flash	StepFun · Open weight	Artificial Analysis τ²-bench	87.4%
49	GPT-5.4	OpenAI · Closed	τ²-bench Telecom	87.1%
50	Gemini 3 Pro	Google · Closed	Artificial Analysis τ²-bench	87.1%
51	GPT-5 (medium)	OpenAI · Closed	Artificial Analysis τ²-bench	86.5%
52	GPT-5.6 Terra	OpenAI · Closed	Artificial Analysis τ²-bench	86.3%
53	Claude Opus 4.5	Anthropic · Closed	Artificial Analysis τ²-bench	86.3%
54	GPT-5.3 Codex	OpenAI · Closed	Artificial Analysis τ²-bench	86%
55	Ling 2.6 Flash	InclusionAI · Open weight	Artificial Analysis τ²-bench	86%
56	GPT-5.3-Codex-Spark	OpenAI · Closed	Artificial Analysis τ²-bench	86%
57	GPT-5.6 Sol	OpenAI · Closed	Artificial Analysis τ²-bench	85.1%
58	Claude Opus 4.6	Anthropic · Closed	Artificial Analysis τ²-bench	84.8%
59	GPT-5.2	OpenAI · Closed	Artificial Analysis τ²-bench	84.8%
60	MiniMax M2.7	MiniMax · Open weight	Artificial Analysis τ²-bench	84.8%
61	GPT-5 (high)	OpenAI · Closed	Artificial Analysis τ²-bench	84.8%
62	MiMo-V2-Flash	Xiaomi · Open weight	Artificial Analysis τ²-bench	83.9%
63	GPT-5.4 mini	OpenAI · Closed	τ²-bench Telecom	83.3%
64	Nemotron 3 Ultra	NVIDIA · Open weight	Artificial Analysis τ²-bench	83.3%
65	GPT-5.1-Codex-Max	OpenAI · Closed	Artificial Analysis τ²-bench	83%
66	GPT-5.1-Codex	OpenAI · Closed	Artificial Analysis τ²-bench	83%
67	GPT-5.1	OpenAI · Closed	Artificial Analysis τ²-bench	81.9%
68	o3	OpenAI · Closed	Artificial Analysis τ²-bench	80.7%
69	Command A+	Cohere · Open weight	τ²-bench Telecom	80.7%
70	Claude Sonnet 4.6	Anthropic · Closed	Artificial Analysis τ²-bench	79.5%
71	DeepSeek V3.2	DeepSeek · Open weight	Artificial Analysis τ²-bench	78.9%
72	GLM-4.6	Z.AI · Open weight	Artificial Analysis τ²-bench	76.9%
73	GPT-5.4 nano	OpenAI · Closed	τ²-bench Telecom	76%
74	Grok Code Fast 1	xAI · Closed	Artificial Analysis τ²-bench	75.7%
75	Grok 4	xAI · Closed	Artificial Analysis τ²-bench	74.9%
76	Qwen3 Max	Alibaba · Closed	Artificial Analysis τ²-bench	74.3%
77	K-Exaone	LG AI Research · Closed	Artificial Analysis τ²-bench	74.3%
78	Claude Opus 4.7	Anthropic · Closed	Artificial Analysis τ²-bench	74%
79	Claude 4.1 Opus Thinking	Anthropic · Closed	Artificial Analysis τ²-bench	71.4%
80	Mercury 2	Inception · Closed	Artificial Analysis τ²-bench	70.8%
81	GPT-5 mini	OpenAI · Closed	Artificial Analysis τ²-bench	68.4%
82	Nemotron 3 Super 120B A12B	NVIDIA · Open weight	Artificial Analysis τ²-bench	67.8%
83	Grok 4 Fast (Reasoning)	xAI · Closed	Artificial Analysis τ²-bench	65.8%
84	GPT-OSS 120B	OpenAI · Open weight	Artificial Analysis τ²-bench	65.8%
85	Grok 4.1 Fast	xAI · Closed	Artificial Analysis τ²-bench	63.7%
86	o1	OpenAI · Closed	Artificial Analysis τ²-bench	62.6%
87	Kimi K2	Moonshot AI · Closed	Artificial Analysis τ²-bench	61.1%
88	GPT-OSS 20B	OpenAI · Open weight	Artificial Analysis τ²-bench	60.2%
89	Gemma 4 31B	Google · Open weight	Artificial Analysis τ²-bench	59.9%
90	Gemini 2.5 Pro	Google · Closed	Artificial Analysis τ²-bench	54.1%
91	GPT-4.1 mini	OpenAI · Closed	Artificial Analysis τ²-bench	52.9%
92	Claude 4 Sonnet	Anthropic · Closed	Artificial Analysis τ²-bench	52.3%
93	GPT-4.1	OpenAI · Closed	Artificial Analysis τ²-bench	47.1%
94	Sarvam 105B	Sarvam · Open weight	Artificial Analysis τ²-bench	46.8%
95	GLM-4.5-Air	Z.AI · Closed	Artificial Analysis τ²-bench	46.5%
96	Nemotron 3 Nano Omni 30B A3B	NVIDIA · Open weight	τ²-bench Telecom	45.3%
97	Gemma 4 26B A4B	Google · Open weight	Artificial Analysis τ²-bench	43.6%
98	Gemini 3 Flash	Google · Closed	Artificial Analysis τ²-bench	43.3%
99	Mistral Small 4	Mistral · Open weight	Artificial Analysis τ²-bench	41.2%
100	Mistral Small 4 (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	41.2%
101	Nemotron 3 Nano 30B	NVIDIA · Open weight	Artificial Analysis τ²-bench	40.9%
102	DeepSeek V3.1 (Reasoning)	DeepSeek · Open weight	Artificial Analysis τ²-bench	37.4%
103	DeepSeek-R1	DeepSeek · Open weight	Artificial Analysis τ²-bench	36.5%
104	GPT-5 nano	OpenAI · Closed	Artificial Analysis τ²-bench	36.5%
105	Gemma 4 12B	Google · Open weight	Artificial Analysis τ²-bench	36.3%
106	DeepSeek V3.1	DeepSeek · Open weight	Artificial Analysis τ²-bench	34.8%
107	Sarvam 30B	Sarvam · Open weight	Artificial Analysis τ²-bench	34.5%
108	MiniMax M1 80k	MiniMax · Closed	Artificial Analysis τ²-bench	34.2%
109	Solar Pro 2	Upstage · Closed	Artificial Analysis τ²-bench	31.9%
110	Gemini 3.1 Flash-Lite	Google · Closed	Artificial Analysis τ²-bench	31.3%
111	Mistral Large 2	Mistral · Closed	Artificial Analysis τ²-bench	30.7%
112	o3-mini	OpenAI · Closed	Artificial Analysis τ²-bench	28.7%
113	Ministral 3 14B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	27.2%
114	Ministral 3 14B	Mistral · Open weight	Artificial Analysis τ²-bench	27.2%
115	Ministral 3 8B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	26.6%
116	Ministral 3 8B	Mistral · Open weight	Artificial Analysis τ²-bench	26.6%
117	GPT-4o	OpenAI · Closed	Artificial Analysis τ²-bench	25.1%
118	Ministral 3 3B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	24.9%
119	Ministral 3 3B	Mistral · Open weight	Artificial Analysis τ²-bench	24.9%
120	Mistral Large 3	Mistral · Closed	Artificial Analysis τ²-bench	24.6%
121	Mistral Medium 3	Mistral · Closed	Artificial Analysis τ²-bench	24.3%
122	DeepSeek V3	DeepSeek · Open weight	Artificial Analysis τ²-bench	22.8%
123	Granite-4.0-1B	IBM · Open weight	Artificial Analysis τ²-bench	22.8%
124	Claude 3 Haiku	Anthropic · Closed	Artificial Analysis τ²-bench	21.1%
125	Gemma 4 E2B	Google · Open weight	Artificial Analysis τ²-bench	20.8%
126	Gemma 4 E4B	Google · Open weight	Artificial Analysis τ²-bench	20.8%
127	Exaone 4.0 1.2B	LG AI Research · Open weight	Artificial Analysis τ²-bench	20.5%
128	Granite-4.0-H-1B	IBM · Open weight	Artificial Analysis τ²-bench	19.6%
129	LFM2.5-1.2B-Thinking	LiquidAI · Closed	Artificial Analysis τ²-bench	19.6%
130	Llama 3.1 405B	Meta · Open weight	Artificial Analysis τ²-bench	19%
131	Llama 4 Maverick	Meta · Open weight	Artificial Analysis τ²-bench	17.8%
132	GPT-4.1 nano	OpenAI · Closed	Artificial Analysis τ²-bench	17.3%
133	LFM2.5-8B-A1B	LiquidAI · Open weight	τ²-bench Telecom	16.1%
134	Llama 4 Scout	Meta · Open weight	Artificial Analysis τ²-bench	15.5%
135	Gemini 2.5 Flash	Google · Closed	Artificial Analysis τ²-bench	14.9%
136	Granite-4.0-H-350M	IBM · Open weight	Artificial Analysis τ²-bench	14.6%
137	Nova Pro	Amazon · Closed	Artificial Analysis τ²-bench	14%
138	Granite-4.0-350M	IBM · Open weight	Artificial Analysis τ²-bench	13.2%
139	Nemotron Ultra 253B	NVIDIA · Open weight	Artificial Analysis τ²-bench	11.4%
140	LFM2-24B-A2B	LiquidAI · Closed	Artificial Analysis τ²-bench	11.1%
141	LFM2.5-1.2B-Instruct	LiquidAI · Closed	Artificial Analysis τ²-bench	10.8%
142	Gemma 3 27B	Google · Open weight	Artificial Analysis τ²-bench	10.5%
143	LFM2.5-VL-1.6B-Extract	LiquidAI · Open weight	Artificial Analysis τ²-bench	8.5%
144	Exaone 4.0 32B	LG AI Research · Open weight	Artificial Analysis τ²-bench	4.1%
145	Phi-4	Microsoft · Open weight	Artificial Analysis τ²-bench	0%