deep Swe

25 models evaluated

#	Model	Provider	Type	Score
4	GPT-5.6 Sol	OpenAI · Closed	mini-swe-agent · max reasoning	72.7%
5	Claude Fable 5	Anthropic · Closed	mini-swe-agent · max reasoning	69.7%
6	GPT-5.6 Terra	OpenAI · Closed	mini-swe-agent · max reasoning	69.6%
7	GLM-5.3	Z.AI · Open weight	mini-swe-agent · max reasoning	69.0%
8	Kimi K3	Moonshot AI · Closed	mini-swe-agent · max reasoning	68.5%
9	GPT-5.6 Luna	OpenAI · Closed	mini-swe-agent · max reasoning	67.2%
10	GPT-5.5	OpenAI · Closed	mini-swe-agent · xhigh reasoning	67.0%
11	Grok 4.6	xAI · Closed	mini-swe-agent · xhigh reasoning	66.7%
12	Gemini 3.7 Flash	Google · Closed	mini-swe-agent · high reasoning	65.3%
13	GLM-5.3-Flash	Z.AI · Open weight	mini-swe-agent · max reasoning	63.4%
14	DeepSeek V4 Pro 0813	DeepSeek · Closed	mini-swe-agent · max reasoning	62.8%
15	Claude Opus 4.8	Anthropic · Closed	mini-swe-agent · max reasoning	59.0%
16	Qwen3.8 Max	Alibaba · Open weight	mini-swe-agent · xhigh reasoning	57.5%
17	Muse Spark 1.2	Meta · Closed	mini-swe-agent · xhigh reasoning	54.9%
18	Claude Sonnet 5	Anthropic · Closed	mini-swe-agent · max reasoning	53.8%
19	Grok 4.5	xAI · Closed	mini-swe-agent · high reasoning	53.8%
20	DeepSeek V4 Flash 0731	DeepSeek · Closed	mini-swe-agent · max reasoning	53.3%
21	Muse Spark 1.1	Meta · Closed	mini-swe-agent · xhigh reasoning	53.3%
22	GPT-5.4	OpenAI · Closed	mini-swe-agent · xhigh reasoning	51.8%
23	Gemini 3.6 Flash	Google · Closed	mini-swe-agent · high reasoning	46.7%
24	GLM-5.2	Z.AI · Open weight	mini-swe-agent · max reasoning	43.8%
25	Gemini 3.5 Flash	Google · Closed	mini-swe-agent · high reasoning	36.1%
26	Kimi K2.7 Code	Moonshot AI · Open weight	mini-swe-agent	30.5%
27	Claude Sonnet 4.6	Anthropic · Closed	mini-swe-agent · high reasoning	29.9%
28	Gemini 3.1 Pro	Google · Closed	mini-swe-agent · high reasoning	11.7%