lisan Bench

143 models evaluated

#	Model	Provider	Type	Score
3	Anthropic	anthropic/claude-opus-4.6:thinking-16k	Opus 4.6 (16k)	3526.49
4	GPT 5.5 (medium)	OpenAI · Closed	3315.52
5	Sonnet 4.6 (16k)	Anthropic	2944.27
6	GPT 5.4 (medium)	OpenAI · Closed	2738.16
7	Opus 4.8 (high)	Anthropic · Closed	2693.67
8	Opus 4.5 (16k)	Anthropic	2204.43
9	Gemini 3.1 Pro Preview (high)	Google · Closed	1929.11
10	Grok 4 (medium)	xAI · Closed	1778.40
11	Sonnet 5 (high)	Anthropic · Closed	1736.56
12	O3 (medium)	OpenAI · Closed	1523.00
13	Deepseek V3.2 Speciale (thinking)	DeepSeek	1510.70
14	Grok 4.20 Beta (thinking)	xAI · Closed	1464.63
15	GPT 5.2 (medium)	OpenAI · Closed	1458.80
16	GPT 5 (medium)	OpenAI · Closed	1457.28
17	GPT 5.6 Sol (medium)	OpenAI · Closed	1403.28
18	GPT 5.6 Terra (medium)	OpenAI · Closed	1196.90
19	Gemini 3 Pro Preview (high)	Google · Closed	1130.63
20	Gemini 3.5 Flash (high)	Google · Closed	1128.26
21	Sonnet 4.5 (16k)	Anthropic	1090.82
22	Deepseek V4 Flash (high)	DeepSeek · Open weight	1063.47
23	Deepseek V4 Pro (high)	DeepSeek · Open weight	1059.51
24	Deepseek V3.2 (thinking)	DeepSeek · Open weight	925.31
26	Step 3.5 Flash (thinking)	StepFun · Open weight	811.21
27	Grok 4 Fast (thinking)	xAI	806.48
28	GPT 5 Mini (medium)	OpenAI · Closed	758.84
29	GPT 5.6 Luna (medium)	OpenAI · Closed	648.18
30	Kimi K2.5 (thinking)	Moonshot AI · Closed	641.96
31	Kimi K2 (thinking)	Moonshot AI · Closed	633.05
32	GPT 5 Nano (medium)	OpenAI · Closed	626.86
33	Grok 4.1 Fast (thinking)	xAI · Closed	604.43
34	Sonnet 4 (16k)	Anthropic	602.95
35	GLM 5.2 (high)	Z.AI · Open weight	591.99
36	Gemini 3 Flash Preview (high)	Google · Closed	591.88
37	GPT 5.4 Mini (medium)	OpenAI · Closed	591.48
38	GPT 5.4 Nano (medium)	OpenAI · Closed	543.21
39	O3 Mini (medium)	OpenAI · Closed	518.37
40	Doubao Seed 2.0 Pro (thinking)	StepFun	456.93
41	GPT-OSS-120B (medium)	OpenAI · Open weight	448.33
42	Qwen3.5 397B A17B (thinking)	Alibaba · Open weight	387.74
43	O4 Mini (medium)	OpenAI · Closed	352.68
44	GLM 5 (thinking)	Z.AI · Open weight	336.34
45	GPT 5.6 Sol	OpenAI	332.46
46	GPT 5.5	OpenAI	305.30
48	Doubao Seed 2.0 Lite (thinking)	StepFun	265.12
49	Opus 4	Anthropic	262.56
50	Doubao Seed 1.8 (thinking)	StepFun	255.84
51	Minimax M2.5 (thinking)	MiniMax · Closed	228.38
52	Qwen3 235B A22B 2507 (thinking)	Alibaba · Open weight	226.22
53	Opus 4.7	Anthropic · Closed	217.52
54	Opus 4.1	Anthropic · Closed	215.66
55	Sonnet 4.6	Anthropic · Closed	208.64
56	Sonnet 5	Anthropic	208.26
57	Gemini 2.5 Pro (16k)	Google · Closed	197.43
58	Grok 3 Mini (medium)	xAI · Closed	194.03
59	Grok 3 (thinking)	xAI · Closed	188.12
60	Sonnet 3.7	Anthropic	162.44
61	GPT-OSS-20B (medium)	OpenAI · Open weight	156.99
62	GPT 5.6 Terra	OpenAI	156.14
63	Doubao Seed 2.0 Mini (thinking)	StepFun	150.35
64	Sonnet 4	Anthropic · Closed	150.17
65	Sonnet 3.6	Anthropic · Closed	149.65
67	Gemini Pro 1.5	Google · Closed	119.72
68	Deepseek V3.2	DeepSeek · Open weight	119.04
69	Deepseek V4 Pro	DeepSeek · Open weight	117.14
70	Gemini 2.5 Flash (16k)	Google	112.75
71	Deepseek R1 0528 (thinking)	DeepSeek · Open weight	111.60
72	Qwen3.5 122B A10B (thinking)	Alibaba · Open weight	109.62
74	GLM 4.5 (thinking)	Z.AI · Closed	108.32
75	Qwen3.5 35B A3B (thinking)	Alibaba · Open weight	107.61
76	Olmo 3 32B (thinking)	Allen AI	104.86
77	Sonnet 4.5	Anthropic · Closed	103.58
78	Deepseek V3	DeepSeek · Open weight	103.39
79	O1 Mini (medium)	OpenAI	103.10
80	GPT 5.6 Luna	OpenAI	96.63
81	GPT 4o	OpenAI · Closed	94.16
82	Opus 4.5	Anthropic · Closed	93.49
83	Opus 4.6	Anthropic · Closed	91.61
84	GPT 4 Turbo	OpenAI · Closed	91.48
86	Qwen3 4B (16k)	Alibaba	77.21
87	Opus 3	Anthropic · Closed	75.77
88	Gemini 2.5 Flash	Google · Closed	72.17
89	Minimax M1 (thinking)	MiniMax · Closed	66.71
90	GLM 5.2	Z.AI	64.50
91	Gemini 2.0 Flash	Google	62.04
92	Deepseek V4 Flash	DeepSeek · Open weight	56.78
93	Horizon Beta	OpenRouter	55.69
94	Gemini Flash 1.5	Google	55.20
95	GLM 4.5 Air (thinking)	Z.AI · Closed	54.86
96	Nova Pro V1	Amazon · Closed	54.38
97	GLM 4.7 (thinking)	Z.AI · Open weight	54.27
98	Polaris Alpha	OpenRouter	53.34
99	Haiku 4.5	Anthropic · Closed	52.64
100	GPT 3.5 Turbo	OpenAI	50.96
101	Qwen3 Coder	Alibaba	50.95
102	Llama 3.1 405B	Meta	48.88
103	Grok 4.1 Fast	xAI · Closed	47.29
104	GLM 4.6 (thinking)	Z.AI · Open weight	44.20
105	Gemma 3 27B	Google	43.79
106	Mistral Medium 3	Mistral · Closed	43.31
107	GPT 5.4 Mini	OpenAI	42.74
108	Llama 4 Maverick	Meta · Open weight	42.33
109	GPT 4.1	OpenAI · Closed	42.02
110	Ernie 4.5 300B A47B	Baidu	40.99
111	Sherlock Dash Alpha	OpenRouter	40.07
112	Devstral Medium	Mistral AI	40.03
113	Gemini 2.0 Flash Lite 001	Google	38.77
114	Haiku 3.5	Anthropic	38.22
115	Gemini 2.5 Flash Lite	Google	38.21
116	Llama 3.1 70B	Meta	38.06
117	Qwen3 1.7B (16k)	Alibaba	38.06
118	Haiku 3	Anthropic · Closed	35.46
119	Mistral Large 2411	Mistral AI	34.64
120	GPT 4.1 Mini	OpenAI · Closed	32.82
121	Gemini 2.5 Flash Lite (16k)	Google	32.37
122	Qwen3 235B A22B 2507	Alibaba	31.92
123	Llama 4 Scout	Meta · Open weight	30.85
124	Mimo V2 Flash (thinking)	Xiaomi · Open weight	28.75
125	Nova Lite V1	Amazon	25.24
126	Nova Micro V1	Amazon	24.88
127	Gemini Flash 1.5 8B	Google	24.38
128	Qwen3 32B	Alibaba	23.86
129	Gemma 3 12B	Google	21.56
130	GPT 4o Mini	OpenAI · Closed	21.21
131	Qwen3 30B A3B 2507	Alibaba	18.98
132	Mistral Small 3.2 24B	Mistral AI	17.77
133	Qwen3 14B	Alibaba	16.68
134	Qwen3 8B	Alibaba	15.50
135	GPT 4.1 Nano	OpenAI · Closed	14.95
136	Devstral Small	Mistral AI	13.22
137	Codestral 2508	Mistral AI	13.15
138	Ministral 14B 2512	Mistral AI	12.29
139	Ministral 8B 2512	Mistral AI	11.78
140	Mistral Nemo	Mistral AI	11.47
141	GPT 5.4 Nano	OpenAI	11.16
142	Gemma 3 4B	Google	9.41
143	Qwen3 0.6B (16k)	Alibaba	9.24
144	Qwen3 4B	Alibaba	7.90
145	Ministral 3B 2512	Mistral AI	6.64
146	Qwen3 1.7B	Alibaba	6.27
147	Llama 3.1 8B	Meta	3.92
148	Llama 3.2 3B	Meta	2.85
149	Llama 3.2 1B	Meta	0.63
150	Qwen3 0.6B	Alibaba	0.06