lisan Bench

135 models evaluated

#	Model	Provider	Type	Score
1	Claude Opus 4.7 (Adaptive)	Anthropic	Proprietary	5122.60
2	Claude Opus 5	Anthropic	Proprietary	5068.25
3	Claude Fable 5	Anthropic	Proprietary	4561.82
4	Claude Opus 4.6 (Adaptive)	Anthropic · Closed	3526.49
5	Kimi K3	Moonshot AI · Closed	3525.77
6	GPT-5.5	OpenAI · Closed	3315.52
7	Claude Sonnet 4.6	Anthropic · Closed	2944.27
8	GPT-5.4	OpenAI · Closed	2738.16
9	Claude Opus 4.8	Anthropic · Closed	2693.67
11	Claude Opus 4.5 Thinking	Anthropic · Closed	2204.43
12	Gemini 3.1 Pro	Google · Closed	1929.11
13	Grok 4	xAI · Closed	1778.40
14	Claude Sonnet 5	Anthropic · Closed	1736.56
15	o3	OpenAI · Closed	1523.00
16	Deepseek V3.2 Speciale (thinking)	DeepSeek	1510.70
17	Grok 4.20	xAI · Closed	1464.63
18	GPT-5.2	OpenAI · Closed	1458.80
19	GPT-5 (medium)	OpenAI · Closed	1457.28
20	GPT-5.6 Sol	OpenAI · Closed	1403.28
21	GPT-5.6 Terra	OpenAI · Closed	1196.90
22	Gemini 3 Pro	Google · Closed	1130.63
23	Gemini 3.5 Flash	Google · Closed	1128.26
24	Claude Sonnet 4.5 Thinking	Anthropic · Closed	1090.82
25	DeepSeek V4 Flash 0731	DeepSeek · Closed	1063.47
26	DeepSeek V4 Pro 0813	DeepSeek · Closed	1059.51
27	DeepSeek V3.2 (Thinking)	DeepSeek · Open weight	925.31
29	Step 3.5 Flash	StepFun · Open weight	811.21
30	Grok 4 Fast (Reasoning)	xAI · Closed	806.48
31	GPT-5 mini	OpenAI · Closed	758.84
32	GPT-5.6 Luna	OpenAI · Closed	648.18
33	Kimi K2.5 (Reasoning)	Moonshot AI · Closed	641.96
34	Kimi K2	Moonshot AI · Closed	633.05
35	GPT-5 nano	OpenAI · Closed	626.86
36	Grok 4.1 Fast (Reasoning)	xAI · Closed	604.43
37	Sonnet 4 (16k)	Anthropic	602.95
38	GLM-5.2	Z.AI · Open weight	591.99
39	Gemini 3 Flash	Google · Closed	591.88
40	GPT-5.4 mini	OpenAI · Closed	591.48
41	GPT-5.4 nano	OpenAI · Closed	543.21
42	o3-mini	OpenAI · Closed	518.37
43	Doubao Seed 2.0 Pro (thinking)	StepFun	456.93
44	GPT-OSS 120B	OpenAI · Open weight	448.33
45	Qwen3.5 397B (Reasoning)	Alibaba · Open weight	387.74
47	o4-mini (high)	OpenAI · Closed	352.68
48	GLM-5 (Reasoning)	Z.AI · Open weight	336.34
52	Doubao Seed 2.0 Lite (thinking)	StepFun	265.12
53	Opus 4	Anthropic	262.56
54	Doubao Seed 1.8 (thinking)	StepFun	255.84
55	MiniMax M2.5	MiniMax · Closed	228.38
56	Qwen3 235B 2507 (Reasoning)	Alibaba · Open weight	226.22
57	Claude Opus 4.7	Anthropic · Closed	217.52
58	Claude 4.1 Opus	Anthropic · Closed	215.66
61	Gemini 2.5 Pro	Google · Closed	197.43
62	Grok 3 Mini	xAI · Closed	194.03
63	Grok 3 [Beta]	xAI · Closed	188.12
64	Sonnet 3.7	Anthropic	162.44
65	GPT-OSS 20B	OpenAI · Open weight	156.99
67	Doubao Seed 2.0 Mini (thinking)	StepFun	150.35
68	Claude 4 Sonnet	Anthropic · Closed	150.17
69	Claude 3.5 Sonnet	Anthropic · Closed	149.65
71	Gemini 1.5 Pro	Google · Closed	119.72
72	DeepSeek V3.2	DeepSeek · Open weight	119.04
74	Gemini 2.5 Flash	Google · Closed	112.75
75	DeepSeek-R1	DeepSeek · Open weight	111.60
76	Qwen3.5-122B-A10B	Alibaba · Open weight	109.62
78	GLM-4.5	Z.AI · Closed	108.32
79	Qwen3.5-35B-A3B	Alibaba · Open weight	107.61
80	Olmo 3 32B (thinking)	Allen AI	104.86
81	Claude Sonnet 4.5	Anthropic · Closed	103.58
82	DeepSeek V3	DeepSeek · Open weight	103.39
83	O1 Mini (medium)	OpenAI	103.10
85	GPT-4o	OpenAI · Closed	94.16
86	Claude Opus 4.5	Anthropic · Closed	93.49
87	Claude Opus 4.6	Anthropic · Closed	91.61
88	GPT-4 Turbo	OpenAI · Closed	91.48
90	Qwen3 4B (16k)	Alibaba	77.21
91	Claude 3 Opus	Anthropic · Closed	75.77
93	MiniMax M1 80k	MiniMax · Closed	66.71
95	Gemini 2.0 Flash	Google	62.04
97	Horizon Beta	OpenRouter	55.69
98	Gemini Flash 1.5	Google	55.20
99	GLM-4.5-Air	Z.AI · Closed	54.86
100	Nova Pro	Amazon · Closed	54.38
101	GLM-4.7	Z.AI · Open weight	54.27
102	Polaris Alpha	OpenRouter	53.34
103	Claude Haiku 4.5	Anthropic · Closed	52.64
104	GPT 3.5 Turbo	OpenAI	50.96
105	Qwen3 Coder	Alibaba	50.95
106	Llama 3.1 405B	Meta · Open weight	48.88
107	Grok 4.1 Fast	xAI · Closed	47.29
108	GLM-4.6	Z.AI · Open weight	44.20
109	Gemma 3 27B	Google · Open weight	43.79
110	Mistral Medium 3	Mistral · Closed	43.31
112	Llama 4 Maverick	Meta · Open weight	42.33
113	GPT-4.1	OpenAI · Closed	42.02
114	Ernie 4.5 300B A47B	Baidu	40.99
115	Sherlock Dash Alpha	OpenRouter	40.07
116	Devstral Medium	Mistral AI	40.03
117	Gemini 2.0 Flash Lite 001	Google	38.77
118	Haiku 3.5	Anthropic	38.22
119	Gemini 2.5 Flash Lite	Google	38.21
120	Llama 3.1 70B	Meta	38.06
121	Qwen3 1.7B (16k)	Alibaba	38.06
122	Claude 3 Haiku	Anthropic · Closed	35.46
123	Mistral Large 2411	Mistral AI	34.64
124	GPT-4.1 mini	OpenAI · Closed	32.82
125	Gemini 2.5 Flash Lite (16k)	Google	32.37
126	Qwen3 235B A22B 2507	Alibaba	31.92
127	Llama 4 Scout	Meta · Open weight	30.85
128	MiMo-V2-Flash	Xiaomi · Open weight	28.75
129	Nova Lite V1	Amazon	25.24
130	Nova Micro V1	Amazon	24.88
131	Gemini Flash 1.5 8B	Google	24.38
132	Qwen3 32B	Alibaba	23.86
133	Gemma 3 12B	Google	21.56
134	GPT-4o mini	OpenAI · Closed	21.21
135	Qwen3 30B A3B 2507	Alibaba	18.98
136	Mistral Small 3.2 24B	Mistral AI	17.77
137	Qwen3 14B	Alibaba	16.68
138	Qwen3 8B	Alibaba	15.50
139	GPT-4.1 nano	OpenAI · Closed	14.95
140	Devstral Small	Mistral AI	13.22
141	Codestral 2508	Mistral AI	13.15
142	Ministral 14B 2512	Mistral AI	12.29
143	Ministral 8B 2512	Mistral AI	11.78
144	Mistral Nemo	Mistral AI	11.47
146	Gemma 3 4B	Google	9.41
147	Qwen3 0.6B (16k)	Alibaba	9.24
148	Qwen3 4B	Alibaba	7.90
149	Ministral 3B 2512	Mistral AI	6.64
150	Qwen3 1.7B	Alibaba	6.27
151	Llama 3.1 8B	Meta	3.92
152	Llama 3.2 3B	Meta	2.85
153	Llama 3.2 1B	Meta	0.63
154	Qwen3 0.6B	Alibaba	0.06