tau2 Bench

148 models evaluated

#	Model	Provider	Type	Score
2	GPT-5.4	OpenAI	Proprietary	τ²-bench Telecom
4	Claude Fable 5	Anthropic · Closed	Artificial Analysis τ²-bench	98.5%
5	GLM-5-Turbo	Z.AI · Closed	Artificial Analysis τ²-bench	98.5%
6	GLM-5V-Turbo	Z.AI · Closed	Artificial Analysis τ²-bench	98.5%
7	Step 3.7 Flash	StepFun · Open weight	Artificial Analysis τ²-bench	98.5%
8	GLM-5	Z.AI · Open weight	Artificial Analysis τ²-bench	98.2%
9	GPT-5.5	OpenAI · Closed	τ²-bench Telecom	98%
10	GLM-5.1	Z.AI · Open weight	Artificial Analysis τ²-bench	97.7%
11	Qwen3.6 Plus	Alibaba · Closed	Artificial Analysis τ²-bench	97.7%
12	Grok 4.3	xAI · Closed	Artificial Analysis τ²-bench	97.7%
13	DeepSeek V4 Pro 0813	DeepSeek · Closed	Artificial Analysis τ²-bench	96.2%
14	Kimi K2.6	Moonshot AI · Open weight	Artificial Analysis τ²-bench	95.9%
15	Kimi K2.5	Moonshot AI · Open weight	Artificial Analysis τ²-bench	95.9%
16	GLM-4.7	Z.AI · Open weight	Artificial Analysis τ²-bench	95.9%
17	Qwen 3.6 Max (preview)	Alibaba · Closed	Artificial Analysis τ²-bench	95.9%
18	Kimi K2.5 (Reasoning)	Moonshot AI · Closed	Artificial Analysis τ²-bench	95.9%
19	Gemini 3.1 Pro	Google · Closed	τ²-bench published setup	95.6%
20	Gemini 3.5 Flash	Google · Closed	Artificial Analysis τ²-bench	95.3%
21	Qwen3.6-35B-A3B	Alibaba · Open weight	Artificial Analysis τ²-bench	95.3%
22	MiniMax M2.5	MiniMax · Closed	Artificial Analysis τ²-bench	95.3%
23	MiMo-V2-Pro	Xiaomi · Closed	Artificial Analysis τ²-bench	95%
24	Qwen3.7 Max	Alibaba · Closed	Artificial Analysis τ²-bench	94.7%
25	Claude Opus 4.8	Anthropic · Closed	Artificial Analysis τ²-bench	94.4%
26	Mistral Medium 3.5 128B	Mistral · Open weight	Artificial Analysis τ²-bench	94.2%
27	Qwen3.6-27B	Alibaba · Open weight	Artificial Analysis τ²-bench	94.2%
28	MiMo-V2.5-Pro	Xiaomi · Closed	Artificial Analysis τ²-bench	94.2%
30	Qwen3.5-27B	Alibaba · Open weight	Artificial Analysis τ²-bench	93.9%
31	Qwen3.5-122B-A10B	Alibaba · Open weight	Artificial Analysis τ²-bench	93.6%
32	GPT-5.4 mini	OpenAI · Closed	τ²-bench Telecom	93.4%
33	Grok 4.1 Fast (Reasoning)	xAI · Closed	Artificial Analysis τ²-bench	93.3%
34	Qwen3.7 Plus	Alibaba · Closed	Artificial Analysis τ²-bench	93%
35	GPT-5.4 nano	OpenAI · Closed	τ²-bench Telecom	92.5%
36	GPT-5.2-Codex	OpenAI · Closed	Artificial Analysis τ²-bench	92.1%
37	Claude Opus 4.6 (Adaptive)	Anthropic · Closed	Artificial Analysis τ²-bench	92.1%
38	Muse Spark	Meta · Closed	τ²-bench published setup	91.5%
39	MiMo-V2-Omni	Xiaomi · Closed	Artificial Analysis τ²-bench	91.2%
40	Kimi K2.7 Code	Moonshot AI · Open weight	Artificial Analysis τ²-bench	90.1%
41	Trinity-Large-Thinking	Arcee AI · Open weight	Artificial Analysis τ²-bench	90.1%
42	Trinity-Large-Preview	Arcee AI · Open weight	Artificial Analysis τ²-bench	90.1%
43	Claude Opus 4.5 Thinking	Anthropic · Closed	Artificial Analysis τ²-bench	89.5%
44	Qwen3.5-35B-A3B	Alibaba · Open weight	Artificial Analysis τ²-bench	89.2%
45	MiniMax M3	MiniMax · Open weight	Artificial Analysis τ²-bench	88.9%
46	Claude Opus 4.7 (Adaptive)	Anthropic · Closed	Artificial Analysis τ²-bench	88.6%
47	LFM2.5-8B-A1B	LiquidAI · Open weight	τ²-bench Telecom	88.1%
48	Step 3.5 Flash	StepFun · Open weight	Artificial Analysis τ²-bench	87.4%
49	Gemini 3 Pro	Google · Closed	Artificial Analysis τ²-bench	87.1%
50	GPT-5 (medium)	OpenAI · Closed	Artificial Analysis τ²-bench	86.5%
51	GPT-5.6 Terra	OpenAI · Closed	Artificial Analysis τ²-bench	86.3%
52	Claude Opus 4.5	Anthropic · Closed	Artificial Analysis τ²-bench	86.3%
53	Solar Pro 3	Upstage · Closed	Artificial Analysis τ²-bench	86.3%
54	GPT-5.3 Codex	OpenAI · Closed	Artificial Analysis τ²-bench	86%
55	Ling 2.6 Flash	InclusionAI · Open weight	Artificial Analysis τ²-bench	86%
56	GPT-5.3-Codex-Spark	OpenAI · Closed	Artificial Analysis τ²-bench	86%
57	GPT-5.6 Sol	OpenAI · Closed	Artificial Analysis τ²-bench	85.1%
58	Command A+	Cohere · Open weight	τ²-bench Telecom	85%
59	Claude Opus 4.6	Anthropic · Closed	Artificial Analysis τ²-bench	84.8%
60	MiniMax M2.7	MiniMax · Open weight	Artificial Analysis τ²-bench	84.8%
61	GPT-5.2	OpenAI · Closed	Artificial Analysis τ²-bench	84.8%
62	GPT-5 (high)	OpenAI · Closed	Artificial Analysis τ²-bench	84.8%
63	Qwen3.5 397B	Alibaba · Open weight	Artificial Analysis τ²-bench	83.9%
64	MiMo-V2-Flash	Xiaomi · Open weight	Artificial Analysis τ²-bench	83.9%
65	Qwen3.5 397B (Reasoning)	Alibaba · Open weight	Artificial Analysis τ²-bench	83.9%
66	Nemotron 3 Ultra	NVIDIA · Open weight	Artificial Analysis τ²-bench	83.3%
67	GPT-5.1-Codex	OpenAI · Closed	Artificial Analysis τ²-bench	83%
68	GPT-5.1-Codex-Max	OpenAI · Closed	Artificial Analysis τ²-bench	83%
69	GPT-5.1	OpenAI · Closed	Artificial Analysis τ²-bench	81.9%
70	o3	OpenAI · Closed	Artificial Analysis τ²-bench	80.7%
71	LLaDA2.2-flash	InclusionAI · Open weight	τ²-bench published setup	80.3%
72	Ternary Bonsai 2 27B	Prism ML · Open weight	τ²-bench Telecom	80.2%
73	Claude Sonnet 4.6	Anthropic · Closed	Artificial Analysis τ²-bench	79.5%
74	DeepSeek V3.2	DeepSeek · Open weight	Artificial Analysis τ²-bench	78.9%
75	Agents-A1-4B	InternScience · Open weight	τ²-bench published setup	78.2%
76	GLM-4.6	Z.AI · Open weight	Artificial Analysis τ²-bench	76.9%
77	Grok Code Fast 1	xAI · Closed	Artificial Analysis τ²-bench	75.7%
78	Grok 4	xAI · Closed	Artificial Analysis τ²-bench	74.9%
79	K-Exaone	LG AI Research · Closed	Artificial Analysis τ²-bench	74.3%
80	Qwen3 Max	Alibaba · Closed	Artificial Analysis τ²-bench	74.3%
81	Claude Opus 4.7	Anthropic · Closed	Artificial Analysis τ²-bench	74%
82	Claude 4.1 Opus Thinking	Anthropic · Closed	Artificial Analysis τ²-bench	71.4%
83	Mercury 2	Inception · Closed	Artificial Analysis τ²-bench	70.8%
84	GPT-5 mini	OpenAI · Closed	Artificial Analysis τ²-bench	68.4%
85	Nemotron 3 Super 100B	NVIDIA · Open weight	Artificial Analysis τ²-bench	67.8%
86	Nemotron 3 Super 120B A12B	NVIDIA · Open weight	Artificial Analysis τ²-bench	67.8%
87	Grok 4 Fast (Reasoning)	xAI · Closed	Artificial Analysis τ²-bench	65.8%
88	GPT-OSS 120B	OpenAI · Open weight	Artificial Analysis τ²-bench	65.8%
89	Grok 4.1 Fast	xAI · Closed	Artificial Analysis τ²-bench	63.7%
90	o1	OpenAI · Closed	Artificial Analysis τ²-bench	62.6%
91	Kimi K2	Moonshot AI · Closed	Artificial Analysis τ²-bench	61.1%
92	GPT-OSS 20B	OpenAI · Open weight	Artificial Analysis τ²-bench	60.2%
93	Gemma 4 31B	Google · Open weight	Artificial Analysis τ²-bench	59.9%
94	LLaDA2.2-mini	InclusionAI · Open weight	τ²-bench published setup	57.5%
95	Gemini 2.5 Pro	Google · Closed	Artificial Analysis τ²-bench	54.1%
96	GPT-4.1 mini	OpenAI · Closed	Artificial Analysis τ²-bench	52.9%
97	Claude 4 Sonnet	Anthropic · Closed	Artificial Analysis τ²-bench	52.3%
98	GPT-4.1	OpenAI · Closed	Artificial Analysis τ²-bench	47.1%
99	Sarvam 105B	Sarvam · Open weight	Artificial Analysis τ²-bench	46.8%
100	GLM-4.5-Air	Z.AI · Closed	Artificial Analysis τ²-bench	46.5%
101	Nemotron 3 Nano Omni 30B A3B	NVIDIA · Open weight	τ²-bench Telecom	45.3%
102	Gemma 4 26B A4B	Google · Open weight	Artificial Analysis τ²-bench	43.6%
103	Gemini 3 Flash	Google · Closed	Artificial Analysis τ²-bench	43.3%
104	Mistral Small 4	Mistral · Open weight	Artificial Analysis τ²-bench	41.2%
105	Mistral Small 4 (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	41.2%
106	Nemotron 3 Nano 30B	NVIDIA · Open weight	Artificial Analysis τ²-bench	40.9%
107	DeepSeek V3.1 (Reasoning)	DeepSeek · Open weight	Artificial Analysis τ²-bench	37.4%
108	North Mini Code	Cohere · Open weight	Artificial Analysis τ²-bench	37.4%
109	DeepSeek-R1	DeepSeek · Open weight	Artificial Analysis τ²-bench	36.5%
110	GPT-5 nano	OpenAI · Closed	Artificial Analysis τ²-bench	36.5%
111	Gemma 4 12B	Google · Open weight	Artificial Analysis τ²-bench	36.3%
112	DeepSeek V3.1	DeepSeek · Open weight	Artificial Analysis τ²-bench	34.8%
113	Sarvam 30B	Sarvam · Open weight	Artificial Analysis τ²-bench	34.5%
114	MiniMax M1 80k	MiniMax · Closed	Artificial Analysis τ²-bench	34.2%
115	Solar Pro 2	Upstage · Closed	Artificial Analysis τ²-bench	31.9%
116	Mistral Large 2	Mistral · Closed	Artificial Analysis τ²-bench	30.7%
117	o3-mini	OpenAI · Closed	Artificial Analysis τ²-bench	28.7%
118	Ministral 3 14B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	27.2%
119	Ministral 3 14B	Mistral · Open weight	Artificial Analysis τ²-bench	27.2%
120	Ultravox v0.6 Llama 3.3 70B	Fixie AI · Open weight	Artificial Analysis τ²-bench	26.6%
121	Ministral 3 8B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	26.6%
122	Ministral 3 8B	Mistral · Open weight	Artificial Analysis τ²-bench	26.6%
123	GPT-4o	OpenAI · Closed	Artificial Analysis τ²-bench	25.1%
124	Ministral 3 3B (Reasoning)	Mistral · Open weight	Artificial Analysis τ²-bench	24.9%
125	Ministral 3 3B	Mistral · Open weight	Artificial Analysis τ²-bench	24.9%
126	Mistral Large 3	Mistral · Closed	Artificial Analysis τ²-bench	24.6%
127	Mistral Medium 3	Mistral · Closed	Artificial Analysis τ²-bench	24.3%
128	DeepSeek V3	DeepSeek · Open weight	Artificial Analysis τ²-bench	22.8%
129	Qwen3-Omni-30B-A3B-Thinking	Alibaba · Open weight	Artificial Analysis τ²-bench	21.3%
130	Claude 3 Haiku	Anthropic · Closed	Artificial Analysis τ²-bench	21.1%
131	Gemma 4 E4B	Google · Open weight	Artificial Analysis τ²-bench	20.8%
132	Gemma 4 E2B	Google · Open weight	Artificial Analysis τ²-bench	20.8%
133	Exaone 4.0 1.2B	LG AI Research · Open weight	Artificial Analysis τ²-bench	20.5%
134	Granite-4.0-H-1B	IBM · Open weight	Artificial Analysis τ²-bench	19.6%
135	LFM2.5-1.2B-Thinking	LiquidAI · Closed	Artificial Analysis τ²-bench	19.6%
136	Llama 3.1 405B	Meta · Open weight	Artificial Analysis τ²-bench	19%
137	Llama 4 Maverick	Meta · Open weight	Artificial Analysis τ²-bench	17.8%
138	GPT-4.1 nano	OpenAI · Closed	Artificial Analysis τ²-bench	17.3%
139	Qwen3-Omni-30B-A3B-Instruct	Alibaba · Open weight	Artificial Analysis τ²-bench	16.4%
140	Llama 4 Scout	Meta · Open weight	Artificial Analysis τ²-bench	15.5%
141	Gemini 2.5 Flash	Google · Closed	Artificial Analysis τ²-bench	14.9%
142	Granite-4.0-H-350M	IBM · Open weight	Artificial Analysis τ²-bench	14.6%
143	Nova Pro	Amazon · Closed	Artificial Analysis τ²-bench	14%
144	Granite-4.0-350M	IBM · Open weight	Artificial Analysis τ²-bench	13.2%
145	Nemotron Ultra 253B	NVIDIA · Open weight	Artificial Analysis τ²-bench	11.4%
146	LFM2-24B-A2B	LiquidAI · Closed	Artificial Analysis τ²-bench	11.1%
147	LFM2.5-1.2B-Instruct	LiquidAI · Closed	Artificial Analysis τ²-bench	10.8%
148	Gemma 3 27B	Google · Open weight	Artificial Analysis τ²-bench	10.5%
149	LFM2.5-VL-1.6B-Extract	LiquidAI · Open weight	Artificial Analysis τ²-bench	8.5%
150	Exaone 4.0 32B	LG AI Research · Open weight	Artificial Analysis τ²-bench	4.1%
151	Phi-4	Microsoft · Open weight	Artificial Analysis τ²-bench	0%