Coding tool and agent comparisons

SWE-bench (Verified)

Ranked by resolved on SWE-bench alone. No score from another benchmark is mixed in, and nothing is averaged. How this ranking is made, and what it cannot tell you.

Reference pages for what is ranked here: Qwen, DeepSeek, Llama, Mistral, OpenAI, Anthropic, Google DeepMind

60 results.

PlaceAgent or toolModels usedResolvedRun settingsCost per resolved issue
1mini-SWE-agentSWE-agentclaude-opus-4-5-2025110174.4%
  • attempts: 1
  • reasoning effort: medium
  • checked by SWE-bench: yes
$0.969measured: $360.62 for the run, 372 of 500 solved,
2mini-SWE-agentSWE-agentgemini-3-pro-preview74.2%
  • attempts: 1
  • checked by SWE-bench: yes
$0.62measured: $229.98 for the run, 371 of 500 solved,
3OpenHandsOpenHandsopenai/gpt-5-2025-08-0771.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
4Moatless ToolsMoatless AIclaude-4-sonnet-2025051470.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
5mini-SWE-agentSWE-agentclaude-sonnet-4-5-2025092970.6%
  • attempts: 1
  • checked by SWE-bench: yes
$0.791measured: $279.17 for the run, 353 of 500 solved,
6OpenHandsOpenHandsclaude-4-sonnet-2025051470.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
7mini-SWE-agentSWE-agentclaude-4-opus-2025051467.6%
  • attempts: 1
  • checked by SWE-bench: yes
$1.67measured: $565.64 for the run, 338 of 500 solved,
8SWE-agentSWE-agentclaude-4-sonnet-2025051466.6%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
9OpenHandsOpenHandsUndisclosed65.8%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
10OpenHandsOpenHandsmoonshot/kimi-k2-0711-preview65.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
11mini-SWE-agentSWE-agentgpt-5-2025-08-0765%
  • attempts: 1
  • reasoning effort: medium
  • checked by SWE-bench: yes
$0.431measured: $140.19 for the run, 325 of 500 solved,
12mini-SWE-agentSWE-agentclaude-4-sonnet-2025051464.93%
  • attempts: 1
  • checked by SWE-bench: yes
$0.571measured: $185.73 for the run, 325 of 500 solved,
13mini-SWE-agentSWE-agentKimi-K2-Thinking63.4%
  • attempts: 1
  • checked by SWE-bench: yes
$0.691measured: $219.15 for the run, 317 of 500 solved,
14SWE-agentSWE-agentclaude-3-7-sonnet-2025021962.4%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
15mini-SWE-agentSWE-agentminimax-m261%
  • attempts: 1
  • checked by SWE-bench: yes
$0.702measured: $214.13 for the run, 305 of 500 solved,
16OpenHandsOpenHands4x Scaled60.8%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
17mini-SWE-agentSWE-agentdeepseek-v3.2-reasoner60%
  • attempts: 1
  • checked by SWE-bench: yes
$0.0468measured: $14 for the run, 300 of 500 solved,
18mini-SWE-agentSWE-agentgpt-5-mini-2025-08-0759.8%
  • attempts: 1
  • reasoning effort: medium
  • checked by SWE-bench: yes
$0.0593measured: $17.7 for the run, 299 of 500 solved,
19mini-SWE-agentSWE-agento3-2025041658.4%
  • attempts: 1
  • checked by SWE-bench: yes
$0.571measured: $166.83 for the run, 292 of 500 solved,
20Nemotron-CORTEXANVIDIANV-EmbedCode, claude-3-5-sonnet-20241022, deepseek-v3, o3-mini, gpt-4o-2024-05-13, gpt-4-turbo-2024-04-09, Qwen/Qwen2.5-72B-Instruct, meta-llama/Llama-3.1-405B-Instruct, meta-llama/Llama-3.3-70B-Instruct58.2%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
21mini-SWE-agentSWE-agentdevstral-small-251256.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
22mini-SWE-agentSWE-agentQwen3-Coder-480B-A35B-Instruct55.4%
  • attempts: 1
  • checked by SWE-bench: yes
$0.448measured: $123.96 for the run, 277 of 500 solved,
22mini-SWE-agentSWE-agentglm-4.655.4%
  • attempts: 1
  • checked by SWE-bench: yes
$0.174measured: $48.3 for the run, 277 of 500 solved,
24mini-SWE-agentSWE-agentGLM-4.554.2%
  • attempts: 1
  • checked by SWE-bench: yes
$0.548measured: $148.60 for the run, 271 of 500 solved,
25mini-SWE-agentSWE-agentdevstral-251253.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
26mini-SWE-agentSWE-agentgemini-2.5-pro53.6%
  • attempts: 1
  • checked by SWE-bench: yes
$0.538measured: $144.19 for the run, 268 of 500 solved,
27OpenHandsOpenHandsCodeAct v2.1 (claude-3-5-sonnet-20241022)53%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
28mini-SWE-agentSWE-agentclaude-3-7-sonnet-2025021952.8%
  • attempts: 1
  • checked by SWE-bench: yes
$0.672measured: $177.31 for the run, 264 of 500 solved,
29Composio SWE-Kitclaude-3-5-sonnet-20241022, o1-mini48.6%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
30AppMap Navie v2claude-3-5-sonnet-20241022, gpt-4o-2024-08-0647.2%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
31Skywork-SWE-32BSkywork AIQwen2.5-Coder-32B-Instruct47%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
32OpenHandsOpenHandsmistralai/Devstral-Small-250546.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
33mini-SWE-agentSWE-agento4-mini-2025041645%
  • attempts: 1
  • checked by SWE-bench: yes
$0.467measured: $104.99 for the run, 225 of 500 solved,
34mini-SWE-agentSWE-agentKimi-K2-Instruct43.8%
  • attempts: 1
  • checked by SWE-bench: yes
$1.21measured: $265.80 for the run, 219 of 500 solved,
35Agentless LiteAgentlesso3-mini42.4%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
36SWE-agentSWE-agentQwen 2.540.2%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
37mini-SWE-agentSWE-agentgpt-4.1-2025041439.58%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
38Skywork-SWE-32BSkywork AIQwen2.5-Coder-32B-Instruct38%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
38SWE-agentSWE-agentdevstral-small-250738%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
40mini-SWE-agentSWE-agentgpt-5-nano-2025-08-0734.8%
  • attempts: 1
  • reasoning effort: medium
  • checked by SWE-bench: yes
$0.109measured: $19 for the run, 174 of 500 solved,
41SWE-agentSWE-agentclaude-3-5-sonnet-2024102233.6%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
42SWE-FixerQwen 2.532.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
43mini-SWE-agentSWE-agentgemini-2.5-flash28.73%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
44AppMap Naviegpt-4o-2024-05-1326.2%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
45mini-SWE-agentSWE-agentgpt-oss-120b26%
  • attempts: 1
  • checked by SWE-bench: yes
$0.22measured: $28.6 for the run, 130 of 500 solved,
46mini-SWE-agentSWE-agentgpt-4.1-mini-2025041423.94%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
47SWE-agentSWE-agentgpt-4o-2024-05-1323.2%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
48SWE-agentSWE-agentgpt-4-1106-preview22.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
49mini-SWE-agentSWE-agentgpt-4o-2024112021.62%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
50mini-SWE-agentSWE-agentllama-4-maverick-instruct21.04%
  • attempts: 1
  • checked by SWE-bench: yes
$1.49measured: $156.85 for the run, 105 of 500 solved,
51SWE-agentSWE-agentclaude-3-opus-2024022915.8%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
52mini-SWE-agentSWE-agentgemini-2.0-flash13.52%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
53mini-SWE-agentSWE-agentllama-4-scout-instruct9.06%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
54mini-SWE-agentSWE-agentQwen2.5-Coder-32B-Instruct9%
  • attempts: 1
  • checked by SWE-bench: yes
$0.757measured: $34.1 for the run, 45 of 500 solved,
55RAG baselineclaude-3-opus-202402297%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
56RAG baselineclaude-24.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
57RAG baselinegpt-4-1106-preview2.8%
  • attempts: 2+
  • checked by SWE-bench: yes
Not measured
58RAG baselineSWE-agentSWE-Llama1.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
59RAG baselineSWE-agentSWE-Llama 13B1.2%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured
60RAG baselineSWE-agentGPT-3.50.4%
  • attempts: 1
  • checked by SWE-bench: yes
Not measured

  1. #1 mini-SWE-agent

    Models used
    claude-opus-4-5-20251101
    Resolved
    74.4%,
    Run settings
    attempts: 1; reasoning effort: medium; checked by SWE-bench: yes
    Cost per resolved issue
    $0.969 measured (372 of 500 solved),

  2. #2 mini-SWE-agent

    Models used
    gemini-3-pro-preview
    Resolved
    74.2%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.62 measured (371 of 500 solved),

  3. #3 OpenHands

    Models used
    openai/gpt-5-2025-08-07
    Resolved
    71.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  4. #4 Moatless Tools

    Models used
    claude-4-sonnet-20250514
    Resolved
    70.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  5. #5 mini-SWE-agent

    Models used
    claude-sonnet-4-5-20250929
    Resolved
    70.6%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.791 measured (353 of 500 solved),

  6. #6 OpenHands

    Models used
    claude-4-sonnet-20250514
    Resolved
    70.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  7. #7 mini-SWE-agent

    Models used
    claude-4-opus-20250514
    Resolved
    67.6%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $1.67 measured (338 of 500 solved),

  8. #8 SWE-agent

    Models used
    claude-4-sonnet-20250514
    Resolved
    66.6%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  9. #9 OpenHands

    Models used
    Undisclosed
    Resolved
    65.8%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  10. #10 OpenHands

    Models used
    moonshot/kimi-k2-0711-preview
    Resolved
    65.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  11. #11 mini-SWE-agent

    Models used
    gpt-5-2025-08-07
    Resolved
    65%,
    Run settings
    attempts: 1; reasoning effort: medium; checked by SWE-bench: yes
    Cost per resolved issue
    $0.431 measured (325 of 500 solved),

  12. #12 mini-SWE-agent

    Models used
    claude-4-sonnet-20250514
    Resolved
    64.93%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.571 measured (325 of 500 solved),

  13. #13 mini-SWE-agent

    Models used
    Kimi-K2-Thinking
    Resolved
    63.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.691 measured (317 of 500 solved),

  14. #14 SWE-agent

    Models used
    claude-3-7-sonnet-20250219
    Resolved
    62.4%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  15. #15 mini-SWE-agent

    Models used
    minimax-m2
    Resolved
    61%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.702 measured (305 of 500 solved),

  16. #16 OpenHands

    Models used
    4x Scaled
    Resolved
    60.8%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  17. #17 mini-SWE-agent

    Models used
    deepseek-v3.2-reasoner
    Resolved
    60%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.0468 measured (300 of 500 solved),

  18. #18 mini-SWE-agent

    Models used
    gpt-5-mini-2025-08-07
    Resolved
    59.8%,
    Run settings
    attempts: 1; reasoning effort: medium; checked by SWE-bench: yes
    Cost per resolved issue
    $0.0593 measured (299 of 500 solved),

  19. #19 mini-SWE-agent

    Models used
    o3-20250416
    Resolved
    58.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.571 measured (292 of 500 solved),

  20. #20 Nemotron-CORTEXA

    Models used
    NV-EmbedCode, claude-3-5-sonnet-20241022, deepseek-v3, o3-mini, gpt-4o-2024-05-13, gpt-4-turbo-2024-04-09, Qwen/Qwen2.5-72B-Instruct, meta-llama/Llama-3.1-405B-Instruct, meta-llama/Llama-3.3-70B-Instruct
    Resolved
    58.2%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  21. #21 mini-SWE-agent

    Models used
    devstral-small-2512
    Resolved
    56.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  22. #22 mini-SWE-agent

    Models used
    Qwen3-Coder-480B-A35B-Instruct
    Resolved
    55.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.448 measured (277 of 500 solved),

  23. #22 mini-SWE-agent

    Models used
    glm-4.6
    Resolved
    55.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.174 measured (277 of 500 solved),

  24. #24 mini-SWE-agent

    Models used
    GLM-4.5
    Resolved
    54.2%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.548 measured (271 of 500 solved),

  25. #25 mini-SWE-agent

    Models used
    devstral-2512
    Resolved
    53.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  26. #26 mini-SWE-agent

    Models used
    gemini-2.5-pro
    Resolved
    53.6%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.538 measured (268 of 500 solved),

  27. #27 OpenHands

    Models used
    CodeAct v2.1 (claude-3-5-sonnet-20241022)
    Resolved
    53%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  28. #28 mini-SWE-agent

    Models used
    claude-3-7-sonnet-20250219
    Resolved
    52.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.672 measured (264 of 500 solved),

  29. #29 Composio SWE-Kit

    Models used
    claude-3-5-sonnet-20241022, o1-mini
    Resolved
    48.6%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  30. #30 AppMap Navie v2

    Models used
    claude-3-5-sonnet-20241022, gpt-4o-2024-08-06
    Resolved
    47.2%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  31. #31 Skywork-SWE-32B

    Models used
    Qwen2.5-Coder-32B-Instruct
    Resolved
    47%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  32. #32 OpenHands

    Models used
    mistralai/Devstral-Small-2505
    Resolved
    46.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  33. #33 mini-SWE-agent

    Models used
    o4-mini-20250416
    Resolved
    45%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.467 measured (225 of 500 solved),

  34. #34 mini-SWE-agent

    Models used
    Kimi-K2-Instruct
    Resolved
    43.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $1.21 measured (219 of 500 solved),

  35. #35 Agentless Lite

    Models used
    o3-mini
    Resolved
    42.4%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  36. #36 SWE-agent

    Models used
    Qwen 2.5
    Resolved
    40.2%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  37. #37 mini-SWE-agent

    Models used
    gpt-4.1-20250414
    Resolved
    39.58%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  38. #38 Skywork-SWE-32B

    Models used
    Qwen2.5-Coder-32B-Instruct
    Resolved
    38%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  39. #38 SWE-agent

    Models used
    devstral-small-2507
    Resolved
    38%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  40. #40 mini-SWE-agent

    Models used
    gpt-5-nano-2025-08-07
    Resolved
    34.8%,
    Run settings
    attempts: 1; reasoning effort: medium; checked by SWE-bench: yes
    Cost per resolved issue
    $0.109 measured (174 of 500 solved),

  41. #41 SWE-agent

    Models used
    claude-3-5-sonnet-20241022
    Resolved
    33.6%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  42. #42 SWE-Fixer

    Models used
    Qwen 2.5
    Resolved
    32.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  43. #43 mini-SWE-agent

    Models used
    gemini-2.5-flash
    Resolved
    28.73%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  44. #44 AppMap Navie

    Models used
    gpt-4o-2024-05-13
    Resolved
    26.2%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  45. #45 mini-SWE-agent

    Models used
    gpt-oss-120b
    Resolved
    26%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.22 measured (130 of 500 solved),

  46. #46 mini-SWE-agent

    Models used
    gpt-4.1-mini-20250414
    Resolved
    23.94%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  47. #47 SWE-agent

    Models used
    gpt-4o-2024-05-13
    Resolved
    23.2%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  48. #48 SWE-agent

    Models used
    gpt-4-1106-preview
    Resolved
    22.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  49. #49 mini-SWE-agent

    Models used
    gpt-4o-20241120
    Resolved
    21.62%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  50. #50 mini-SWE-agent

    Models used
    llama-4-maverick-instruct
    Resolved
    21.04%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $1.49 measured (105 of 500 solved),

  51. #51 SWE-agent

    Models used
    claude-3-opus-20240229
    Resolved
    15.8%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  52. #52 mini-SWE-agent

    Models used
    gemini-2.0-flash
    Resolved
    13.52%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  53. #53 mini-SWE-agent

    Models used
    llama-4-scout-instruct
    Resolved
    9.06%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  54. #54 mini-SWE-agent

    Models used
    Qwen2.5-Coder-32B-Instruct
    Resolved
    9%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    $0.757 measured (45 of 500 solved),

  55. #55 RAG baseline

    Models used
    claude-3-opus-20240229
    Resolved
    7%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  56. #56 RAG baseline

    Models used
    claude-2
    Resolved
    4.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  57. #57 RAG baseline

    Models used
    gpt-4-1106-preview
    Resolved
    2.8%,
    Run settings
    attempts: 2+; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  58. #58 RAG baseline

    Models used
    SWE-Llama
    Resolved
    1.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  59. #59 RAG baseline

    Models used
    SWE-Llama 13B
    Resolved
    1.2%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

  60. #60 RAG baseline

    Models used
    GPT-3.5
    Resolved
    0.4%,
    Run settings
    attempts: 1; checked by SWE-bench: yes
    Cost per resolved issue
    Not measured

Source: SWE-bench leaderboard. Each score links the published result it was read from.