aa Sci Code

122 models evaluated

#	Model	Provider	Type	Score
1	Gemini 3.1 Pro	Google	Closed	58.9%
2	GPT-5.4	OpenAI	Closed	56.6%
3	GPT-5.5	OpenAI	Closed	56.1%
4	Gemini 3 Pro	Google	Closed	56.1%
5	GPT-5.2-Codex	OpenAI	Closed	54.6%
6	Claude Opus 4.7 (Adaptive)	Anthropic	Closed	54.5%
7	Kimi K2.6	Moonshot AI	Open	53.5%
8	GPT-5.3 Codex	OpenAI	Closed	53.2%
9	Gemini 3.5 Flash	Google	Closed	53.1%
10	GPT-5.2	OpenAI	Closed	52.1%
11	Claude Opus 4.6 (Adaptive)	Anthropic	Closed	51.9%
12	Muse Spark	Meta	Closed	51.5%
13	MiMo-V2.5-Pro	Xiaomi	Closed	50.2%
14	Claude Opus 4.7	Anthropic	Closed	50.1%
15	DeepSeek V4 Pro (Max)	DeepSeek	Open	50.0%
16	Gemini 3 Flash	Google	Closed	49.9%
17	GPT-5.4 mini	OpenAI	Closed	49.9%
18	Claude Opus 4.5 Thinking	Anthropic	Closed	49.5%
19	Kimi K2.5 (Reasoning)	Moonshot AI	Closed	49.0%
20	Kimi K2.5	Moonshot AI	Open	49.0%
21	Qwen3.7 Max	Alibaba	Closed	48.8%
22	Grok 4.3	xAI	Closed	47.3%
23	Claude Opus 4.5	Anthropic	Closed	47.0%
24	MiniMax M2.7	MiniMax	Open	47.0%
25	Claude Sonnet 4.6	Anthropic	Closed	46.9%
26	Qwen 3.6 Max (preview)	Alibaba	Closed	46.9%
27	GPT-5.4 nano	OpenAI	Closed	46.9%
28	DeepSeek V4 Pro (High)	DeepSeek	Open	46.4%
29	GLM-5	Z.AI	Open	46.2%
30	Claude Opus 4.6	Anthropic	Closed	45.7%
31	Grok 4	xAI	Closed	45.7%
32	GLM-4.7	Z.AI	Open	45.1%
33	DeepSeek V4 Flash (Max)	DeepSeek	Open	44.9%
34	Grok 4.1 Fast (Reasoning)	xAI	Closed	44.2%
35	Grok 4 Fast (Reasoning)	xAI	Closed	44.2%
36	GLM-5.1	Z.AI	Open	43.8%
37	GLM-5-Turbo	Z.AI	Closed	43.6%
38	GLM-5V-Turbo	Z.AI	Closed	43.5%
39	Gemma 4 31B	Google	Open	43.4%
40	GPT-5.1	OpenAI	Closed	43.3%
41	GPT-5 (high)	OpenAI	Closed	42.9%
42	Gemini 2.5 Pro	Google	Closed	42.8%
43	MiMo-V2-Pro	Xiaomi	Closed	42.5%
44	Qwen3.5 397B (Reasoning)	Alibaba	Open	42.0%
45	DeepSeek V4 Flash (High)	DeepSeek	Open	42.0%
46	Qwen3.5-122B-A10B	Alibaba	Open	42.0%
47	Gemini 3.1 Flash-Lite	Google	Closed	41.9%
48	Hy3 Preview	Tencent	Open	41.2%
49	GPT-5 (medium)	OpenAI	Closed	41.1%
50	Qwen3.5 397B	Alibaba	Open	41.1%
51	o3	OpenAI	Closed	41.0%
52	Claude 4.1 Opus Thinking	Anthropic	Closed	40.9%
53	Qwen3.6 Plus	Alibaba	Closed	40.7%
54	GPT-4.1 mini	OpenAI	Closed	40.4%
55	DeepSeek-R1	DeepSeek	Open	40.3%
56	GPT-5.1-Codex-Max	OpenAI	Closed	40.2%
57	GPT-5.1-Codex	OpenAI	Closed	40.2%
58	Gemma 4 26B A4B	Google	Open	40.0%
59	o3-mini	OpenAI	Closed	39.9%
60	Qwen3.6-27B	Alibaba	Open	39.8%
61	Mistral Medium 3.5 128B	Mistral	Open	39.6%
62	Qwen3.5-27B	Alibaba	Open	39.5%
63	DeepSeek V3.1 (Reasoning)	DeepSeek	Open	39.1%
64	GPT-OSS 120B	OpenAI	Open	38.9%
65	DeepSeek V3.2	DeepSeek	Open	38.7%
66	Qwen3 Max	Alibaba	Closed	38.3%
67	GPT-4.1	OpenAI	Closed	38.1%
68	Mistral Small 4 (Reasoning)	Mistral	Open	38.0%
69	Mistral Small 4	Mistral	Open	38.0%
70	Command A+	Cohere	Open	37.8%
71	Qwen3.5-35B-A3B	Alibaba	Open	37.7%
72	DeepSeek R1 Distill Qwen 32B	DeepSeek	Open	37.6%
73	Claude 4 Sonnet	Anthropic	Closed	37.3%
74	DeepSeek V3.1	DeepSeek	Open	36.7%
75	MiMo-V2-Omni	Xiaomi	Closed	36.7%
76	Mistral Large 3	Mistral	Closed	36.2%
77	Grok Code Fast 1	xAI	Closed	36.2%
78	Trinity-Large-Preview	Arcee AI	Open	36.1%
79	Trinity-Large-Thinking	Arcee AI	Open	36.1%
80	Qwen3.6-35B-A3B	Alibaba	Open	35.8%
81	o1	OpenAI	Closed	35.8%
82	K-Exaone	LG AI Research	Closed	35.6%
83	DeepSeek V3	DeepSeek	Open	35.4%
84	Nemotron Ultra 253B	NVIDIA	Open	34.7%
85	Kimi K2	Moonshot AI	Closed	34.5%
86	GPT-OSS 20B	OpenAI	Open	34.4%
87	GPT-4o	OpenAI	Closed	33.3%
88	Llama 4 Maverick	Meta	Open	33.1%
89	Mistral Medium 3	Mistral	Closed	33.1%
90	GLM-4.6	Z.AI	Open	33.1%
91	GPT-4 Turbo	OpenAI	Closed	31.9%
92	GLM-4.5-Air	Z.AI	Closed	30.6%
93	Llama 3.1 405B	Meta	Open	29.9%
94	Grok 4.1 Fast	xAI	Closed	29.6%
95	Gemini 1.5 Pro	Google	Closed	29.5%
96	Mistral Large 2	Mistral	Closed	29.2%
97	Gemini 2.5 Flash	Google	Closed	29.1%
98	Nemotron 3 Nano Omni 30B A3B	NVIDIA	Open	27.8%
99	Ling 2.6 Flash	InclusionAI	Open	27.1%
100	Qwen2.5 Coder 32B Instruct	Alibaba	Open	27.1%
101	Sarvam 105B	Sarvam	Open	26.4%
102	Phi-4	Microsoft	Open	26.0%
103	MiMo-V2-Flash	Xiaomi	Open	25.9%
104	GPT-4.1 nano	OpenAI	Closed	25.9%
105	Exaone 4.0 32B	LG AI Research	Open	25.2%
106	Solar Pro 2	Upstage	Closed	24.8%
107	Gemma 4 E4B	Google	Open	24.4%
108	Claude 3 Opus	Anthropic	Closed	23.3%
109	Nemotron 3 Nano 30B	NVIDIA	Open	23.0%
110	GPT-4o mini	OpenAI	Closed	22.9%
111	Gemma 3 27B	Google	Open	21.2%
112	Gemma 4 E2B	Google	Open	20.9%
113	Nova Pro	Amazon	Closed	20.8%
114	Sarvam 30B	Sarvam	Open	19.2%
115	Claude 3 Haiku	Anthropic	Closed	18.6%
116	Llama 4 Scout	Meta	Open	17.0%
117	Gemini 1.0 Pro	Google	Closed	11.7%
118	Granite-4.0-1B	IBM	Open	8.7%
119	Granite-4.0-H-1B	IBM	Open	8.2%
120	Exaone 4.0 1.2B	LG AI Research	Open	7.4%
121	Granite-4.0-H-350M	IBM	Open	1.7%
122	Granite-4.0-350M	IBM	Open	0.9%