aa Coding Index

119 models evaluated

#	Model	Provider	Type	Score
1	GPT-5.5	OpenAI	Closed	59.1%
2	GPT-5.4	OpenAI	Closed	57.3%
3	Gemini 3.1 Pro	Google	Closed	55.5%
4	GPT-5.3 Codex	OpenAI	Closed	53.1%
5	Claude Opus 4.7	Anthropic	Closed	53.1%
6	Claude Opus 4.7 (Adaptive)	Anthropic	Closed	52.5%
7	GPT-5.4 mini	OpenAI	Closed	51.5%
8	Qwen3.7 Max	Alibaba	Closed	50.1%
9	GPT-5.2	OpenAI	Closed	48.7%
10	Claude Opus 4.6 (Adaptive)	Anthropic	Closed	48.1%
11	Claude Opus 4.5 Thinking	Anthropic	Closed	47.8%
12	Claude Opus 4.6	Anthropic	Closed	47.6%
13	DeepSeek V4 Pro (Max)	DeepSeek	Open	47.5%
14	Muse Spark	Meta	Closed	47.5%
15	Kimi K2.6	Moonshot AI	Open	47.1%
16	Gemini 3 Pro	Google	Closed	46.5%
17	Claude Sonnet 4.6	Anthropic	Closed	46.4%
18	MiMo-V2.5-Pro	Xiaomi	Closed	45.5%
19	Gemini 3.5 Flash	Google	Closed	45.0%
20	Qwen 3.6 Max (preview)	Alibaba	Closed	44.9%
21	GPT-5.1	OpenAI	Closed	44.7%
22	GLM-5	Z.AI	Open	44.2%
23	GPT-5.4 nano	OpenAI	Closed	43.9%
24	GLM-5.1	Z.AI	Open	43.4%
25	DeepSeek V4 Pro (High)	DeepSeek	Open	43.3%
26	GPT-5.2-Codex	OpenAI	Closed	43.0%
27	Claude Opus 4.5	Anthropic	Closed	42.9%
28	Qwen3.6 Plus	Alibaba	Closed	42.9%
29	MiniMax M2.7	MiniMax	Open	41.9%
30	MiMo-V2-Pro	Xiaomi	Closed	41.4%
31	Qwen3.5 397B (Reasoning)	Alibaba	Open	41.3%
32	Grok 4.3	xAI	Closed	41.0%
33	Grok 4	xAI	Closed	40.5%
34	DeepSeek V4 Flash (High)	DeepSeek	Open	39.8%
35	Kimi K2.5 (Reasoning)	Moonshot AI	Closed	39.5%
36	Kimi K2.5	Moonshot AI	Open	39.5%
37	GPT-5 (medium)	OpenAI	Closed	39.0%
38	DeepSeek V4 Flash (Max)	DeepSeek	Open	38.7%
39	Gemma 4 31B	Google	Open	38.7%
40	o3	OpenAI	Closed	38.4%
41	Gemini 3 Flash	Google	Closed	37.8%
42	Qwen3.5 397B	Alibaba	Open	37.4%
43	GLM-5-Turbo	Z.AI	Closed	36.8%
44	GPT-5.1-Codex-Max	OpenAI	Closed	36.6%
45	GPT-5.1-Codex	OpenAI	Closed	36.6%
46	Claude 4.1 Opus Thinking	Anthropic	Closed	36.5%
47	Qwen3.6-27B	Alibaba	Open	36.5%
48	Hy3 Preview	Tencent	Open	36.5%
49	GLM-4.7	Z.AI	Open	36.3%
50	GLM-5V-Turbo	Z.AI	Closed	36.2%
51	GPT-5 (high)	OpenAI	Closed	36.0%
52	MiMo-V2-Omni	Xiaomi	Closed	35.5%
53	Mistral Medium 3.5 128B	Mistral	Open	35.4%
54	Qwen3.6-35B-A3B	Alibaba	Open	35.1%
55	Qwen3.5-27B	Alibaba	Open	34.9%
56	Qwen3.5-122B-A10B	Alibaba	Open	34.7%
57	DeepSeek V3.2	DeepSeek	Open	34.6%
58	o1-preview	OpenAI	Closed	34.0%
59	Gemini 2.5 Pro	Google	Closed	31.9%
60	Grok 4.1 Fast (Reasoning)	xAI	Closed	30.9%
61	Claude 4 Sonnet	Anthropic	Closed	30.6%
62	Qwen3.5-35B-A3B	Alibaba	Open	30.3%
63	GLM-4.6	Z.AI	Open	30.2%
64	Gemini 3.1 Flash-Lite	Google	Closed	30.1%
65	DeepSeek V3.1 (Reasoning)	DeepSeek	Open	29.7%
66	Command A+	Cohere	Open	29.3%
67	GPT-OSS 120B	OpenAI	Open	28.6%
68	DeepSeek V3.1	DeepSeek	Open	28.4%
69	Grok 4 Fast (Reasoning)	xAI	Closed	27.4%
70	Trinity-Large-Preview	Arcee AI	Open	27.2%
71	Trinity-Large-Thinking	Arcee AI	Open	27.2%
72	K-Exaone	LG AI Research	Closed	27.0%
73	Qwen3 Max	Alibaba	Closed	26.4%
74	MiMo-V2-Flash	Xiaomi	Open	25.8%
75	Mistral Small 4 (Reasoning)	Mistral	Open	24.3%
76	Mistral Small 4	Mistral	Open	24.3%
77	DeepSeek-R1	DeepSeek	Open	24.0%
78	GLM-4.5-Air	Z.AI	Closed	23.8%
79	Grok Code Fast 1	xAI	Closed	23.7%
80	Gemini 1.5 Pro	Google	Closed	23.6%
81	Ling 2.6 Flash	InclusionAI	Open	23.2%
82	Mistral Large 3	Mistral	Closed	22.7%
83	Gemma 4 26B A4B	Google	Open	22.4%
84	Kimi K2	Moonshot AI	Closed	22.1%
85	GPT-4.1	OpenAI	Closed	21.8%
86	GPT-4 Turbo	OpenAI	Closed	21.5%
87	o1	OpenAI	Closed	20.5%
88	Claude 3 Opus	Anthropic	Closed	19.5%
89	Grok 4.1 Fast	xAI	Closed	19.5%
90	GPT-OSS 20B	OpenAI	Open	18.5%
91	GPT-4.1 mini	OpenAI	Closed	18.5%
92	o3-mini	OpenAI	Closed	17.9%
93	Gemini 2.5 Flash	Google	Closed	17.8%
94	GPT-4o	OpenAI	Closed	16.7%
95	DeepSeek V3	DeepSeek	Open	16.4%
96	Nemotron 3 Nano 30B	NVIDIA	Open	15.8%
97	Llama 4 Maverick	Meta	Open	15.6%
98	Nemotron 3 Nano Omni 30B A3B	NVIDIA	Open	14.8%
99	Llama 3.1 405B	Meta	Open	14.5%
100	Mistral Large 2	Mistral	Closed	13.8%
101	Gemma 4 E4B	Google	Open	13.7%
102	Mistral Medium 3	Mistral	Closed	13.6%
103	Nemotron Ultra 253B	NVIDIA	Open	13.1%
104	Solar Pro 2	Upstage	Closed	11.3%
105	Phi-4	Microsoft	Open	11.2%
106	GPT-4.1 nano	OpenAI	Closed	11.2%
107	Nova Pro	Amazon	Closed	11.0%
108	Sarvam 105B	Sarvam	Open	9.8%
109	Gemma 3 27B	Google	Open	9.6%
110	Exaone 4.0 32B	LG AI Research	Open	9.4%
111	Gemma 4 E2B	Google	Open	9.0%
112	Sarvam 30B	Sarvam	Open	7.9%
113	Claude 3 Haiku	Anthropic	Closed	6.7%
114	Llama 4 Scout	Meta	Open	6.7%
115	Granite-4.0-1B	IBM	Open	2.9%
116	Granite-4.0-H-1B	IBM	Open	2.7%
117	Exaone 4.0 1.2B	LG AI Research	Open	2.5%
118	Granite-4.0-H-350M	IBM	Open	0.6%
119	Granite-4.0-350M	IBM	Open	0.3%