LongCat-2.5 Benchmarks
The complete evaluation from the official release — 33 tasks across code agents, general agents, and visual coding, compared against 8 frontier and open-weight models
Methodology Notes
- LongCat-2.5 configuration:
enable_thinking=on,temperature=1.0,top_p=0.95unless stated otherwise - Competitor scores marked * are taken from public technical reports, blog posts, or official leaderboards, not re-run by the LongCat team
- Unmarked competitor scores are self-run by the team and are explicitly described as indicative only
- "—" means no number was reported for that model/task pair; it does not imply failure
- Best score per row is bolded; the LongCat-2.5 column is bolded throughout for scanning
Treat launch-day tables as directional. Third-party reproduction lags release by weeks, and several of these benchmarks (Terminal Bench v4.0, Program Bench, SWE Atlas variants, Claw-Eval) are new enough that cross-lab comparability is not fully established.
Code Agent
Ten tasks covering terminal work, repo-scale engineering, test generation, and specification-to-repository builds.
| Benchmark | LongCat-2.5 | LongCat-2.0 | GPT-5.5 | GPT-5.6-Sol | Claude Opus 4.8 | Claude Opus 5 | GLM-5.3 | Kimi-K3 | DeepSeek-V4.1-Flash |
|---|---|---|---|---|---|---|---|---|---|
| Terminal Bench v2.1 | 88.4 | 70.8 | 83.2* | 88.8* | 74.6* | 89.1* | 88.2* | 88.3* | 90.6* |
| Terminal Bench v4.0 | 38.5 | 3.0 | — | 37.3* | 23.6* | 53.9* | 41.8* | 12.6* | 31.2* |
| Program Bench (Almost Resolved) | 25.5 | 2.5 | 13.5* | 23.0* | 16.5* | 37.0* | 19.0* | 17.5* | 20.3* |
| DeepSWE v1.1 | 64.6 | 9.7 | 67.0* | 72.7* | 59.0* | 68.8* | 66.9* | 67.5* | 74.2* |
| PaperBench | 82.1 | 45.2 | 80.0 | 83.3 | 72.9 | 79.1 | 73.7 | 77.5 | — |
| SWE Atlas — Codebase QnA | 58.3 | 33.1 | 45.4* | 46.0* | 57.3* | 63.2* | 58.3 | 54.3 | 55.2 |
| SWE Atlas — Refactoring | 55.2 | 35.7 | 44.8 | — | 46.7 | 51.9 | — | 49.5 | 56.7 |
| SWE Atlas — Test Writing | 47.8 | 28.0 | 42.6 | 45.9 | 49.6 | 62.2 | 42.2 | 42.2 | 48.9 |
| NL2Repo | 60.5 | 40.9 | — | 56.8* | 69.7* | 75.3* | 58.0* | 58.0* | 65.4* |
| LongCat AppDev (internal) | 73.7 | 47.7 | — | 68.8 | — | 76.4 | 73.5 | 75.1 | — |
LongCat AppDev is an internal benchmark, so its numbers are not externally reproducible and should be read as a directional signal only.
General Agent
Fourteen tasks spanning browser research, tool-ecosystem use, long-horizon office automation, professional knowledge work, and skill acquisition.
| Benchmark | LongCat-2.5 | LongCat-2.0 | GPT-5.5 | GPT-5.6-Sol | Claude Opus 4.8 | Claude Opus 5 | GLM-5.3 | Kimi-K3 | DeepSeek-V4.1-Flash |
|---|---|---|---|---|---|---|---|---|---|
| BrowseComp | 91.9 | 79.9 | 84.4* | 90.4* | 84.3* | 90.8* | 81.0 | 91.2* | 91.5 |
| AutomationBench v1.0 | 65.1 | 51.8 | 39.7 | 45.8* | 41.0* | 50.3* | 48.2* | 46.7* | 54.8* |
| HLE (w/ tools) | 63.1 | 37.5 | 52.2 | 58.0* | 57.9* | 64.7* | 62.5* | 56.0* | 63.9* |
| ALE v1.1 | 38.5 | — | — | 30.3* | — | — | 26.2* | 35.2* | — |
| ResearchRubrics | 81.0 | 71.9 | — | 74.2 | — | 72.9 | 79.9 | 78.8 | 80.9 |
| MCP Atlas | 83.1 | 61.8 | 75.3* | 81.8* | 82.2* | 85.8* | 84.2* | 84.2* | 84.4* |
| JobBench | 65.4 | 44.4 | 38.3* | 45.4* | 48.4* | 66.7* | 61.4* | 52.9* | 63.2* |
| SkillsBench | 63.3 | 36.8 | 62.6* | 54.1* | 59.2* | 60.4* | 52.9 | — | 69.8* |
| GDPVal-AA (rubrics) | 84.8 | 72.9 | — | — | — | 87.7 | 87.7 | 86.6 | 86.8 |
| FrontierScience | 77.0 | 66.0 | — | — | — | 78.0 | 75.0 | 70.0 | 73.0 |
| SpreadsheetBench 2 | 34.9 | 24.1 | 30.2 | 33.3 | — | 33.0 | 35.8 | 34.8* | 33.0 |
| FinSearchComp T3 | 63.4 | 42.4 | 62.2 | 62.2 | — | 64.0 | 62.2 | 64.0 | 64.3 |
| Claw-Eval Pass^3 | 78.3 | 67.1 | — | 79.5 | 82.0 | 80.8 | 73.9 | 73.9 | — |
| LongCatClawBench FORTE | 83.3 | 72.2 | 80.7 | 80.6 | 80.6 | 82.6 | 85.6 | 86.3 | 73.5 |
Claw-Eval Pass^3 and LongCatClawBench are harness-specific: they measure performance inside a particular agent scaffold, so they will not transfer one-to-one to your own pipeline.
Visual Coding and Multimodal
Seven tasks. This is the group where 2.5 differs structurally from 2.0 — LongCat-2.0 has no native vision, so its column is empty here by construction.
| Benchmark | LongCat-2.5 | LongCat-2.0 | GPT-5.5 | GPT-5.6-Sol | Claude Opus 4.8 | Claude Opus 5 | GLM-5.3 | Kimi-K3 | DeepSeek-V4.1-Flash |
|---|---|---|---|---|---|---|---|---|---|
| 3DCodeBench (Text→3D) | 55.4 | — | 51.9 | 52.3 | 52.7 | 54.3 | 53.4 | 49.3 | 53.3 |
| Vision2Web | 78.6 | — | 74.6 | 73.5* | 76.1* | 80.2* | 77.8* | 79.0* | 79.2* |
| CharXiv-RQ (w/ tools) | 90.2 | — | 89.0* | 89.1* | 89.9* | 89.3* | — | 91.3* | 87.7* |
| MathVision (w/ tools) | 96.5 | — | 96.8* | 97.8* | 97.1* | — | — | 97.8* | 97.4* |
| VideoMME | 85.2 | — | — | 89.5* | 86.0* | — | — | 90.0* | — |
| PosterBench | 77.4 | — | — | — | 82.3 | — | 82.4 | 76.8 | 75.3 |
| LongCatClawBench-MM FORTE-MM | 72.1 | — | 63.5 | 73.2 | 69.9 | 73.1 | 69.0 | 70.7 | 72.8 |
Where LongCat-2.5 Leads
| Benchmark | LongCat-2.5 | Next best | Margin |
|---|---|---|---|
| BrowseComp | 91.9 | DeepSeek-V4.1-Flash 91.5 | +0.4 |
| AutomationBench v1.0 | 65.1 | DeepSeek-V4.1-Flash 54.8 | +10.3 |
| ALE v1.1 | 38.5 | Kimi-K3 35.2 | +3.3 |
| ResearchRubrics | 81.0 | DeepSeek-V4.1-Flash 80.9 | +0.1 |
| 3DCodeBench (Text→3D) | 55.4 | Claude Opus 5 54.3 | +1.1 |
AutomationBench is the real outlier — a 10-point gap on office-workflow automation is not noise, and it lines up with the release's emphasis on long chains of GUI, spreadsheet, and terminal operations. BrowseComp and ResearchRubrics are wins by a hair and should not be treated as separation. 3DCodeBench is the most interesting result for the modality story: a text-to-3D task won by a model whose vision pipeline is a discrete tokenizer rather than a diffusion adapter.
Where It Trails
| Benchmark | LongCat-2.5 | Leader | Gap |
|---|---|---|---|
| NL2Repo | 60.5 | Claude Opus 5 75.3 | −14.8 |
| Terminal Bench v4.0 | 38.5 | Claude Opus 5 53.9 | −15.4 |
| SWE Atlas — Test Writing | 47.8 | Claude Opus 5 62.2 | −14.4 |
| Program Bench | 25.5 | Claude Opus 5 37.0 | −11.5 |
| DeepSWE v1.1 | 64.6 | DeepSeek-V4.1-Flash 74.2 | −9.6 |
| SkillsBench | 63.3 | DeepSeek-V4.1-Flash 69.8 | −6.5 |
| VideoMME | 85.2 | Kimi-K3 90.0 | −4.8 |
| PosterBench | 77.4 | GLM-5.3 82.4 | −5.0 |
The pattern in the trailing set is consistent: tasks that require building or extending a repository from a specification — NL2Repo, Program Bench, Test Writing, Terminal Bench v4.0 — are exactly where Claude Opus 5 holds a double-digit lead. LongCat-2.5 is strong at executing inside an existing codebase (SWE Atlas Codebase QnA 58.3, Refactoring 55.2, both near the top) and weaker at greenfield construction. If your workload is "add a feature to this 200k-line repo," the gap is small. If it is "build this service from a spec," expect to pay for it in retries.
The Generational Jump
The most defensible claim in the release is internal: 2.5 versus 2.0 on identical benchmarks. These are the same team, same harness, same eval code — the comparison is clean.
| Benchmark | 2.0 | 2.5 | Change |
|---|---|---|---|
| DeepSWE v1.1 | 9.7 | 64.6 | 6.7× |
| Terminal Bench v4.0 | 3.0 | 38.5 | 12.8× |
| Program Bench | 2.5 | 25.5 | 10.2× |
| SkillsBench | 36.8 | 63.3 | +26.5 pts |
| LongCat AppDev | 47.7 | 73.7 | +26.0 pts |
| PaperBench | 45.2 | 82.1 | +36.9 pts |
| HLE (w/ tools) | 37.5 | 63.1 | +25.6 pts |
| MCP Atlas | 61.8 | 83.1 | +21.3 pts |
| BrowseComp | 79.9 | 91.9 | +12.0 pts |
| Terminal Bench v2.1 | 70.8 | 88.4 | +17.6 pts |
A 6–12× improvement on the hardest multi-step coding benchmarks, at constant parameter count, is the whole argument of this release. It says post-training — specifically long-horizon RL with executable verification — is the dominant lever at 1.6T scale, not more pretraining tokens.
Caveat worth stating plainly: 2.0's numbers here are the ones the team re-ran under the current harness. Where 2.0 previously reported a higher figure under a different scaffold (for example Terminal-Bench), the two are not the same measurement.
Which Model Should You Actually Route To
| Workload | Reasonable default | Why |
|---|---|---|
| Terminal / CLI agent looping over a repo | LongCat-2.5 or DeepSeek-V4.1-Flash | Terminal Bench v2.1 is a near-tie (88.4 vs 90.6); decide on cost and latency |
| Building a service from a spec | Claude Opus 5 | NL2Repo +14.8, Program Bench +11.5, Test Writing +14.4 in its favor |
| Screenshot / UI / chart to working code | LongCat-2.5 | Only model in the set with vision in the same token space as the code reasoning |
| Office workflow automation | LongCat-2.5 | AutomationBench +10.3 over the next best; SpreadsheetBench 2 is mid-pack, so test your actual files |
| Deep browsing research | LongCat-2.5 / DeepSeek-V4.1-Flash / Kimi-K3 | All within 0.7 points on BrowseComp — effectively tied |
| Self-hosted / air-gapped deployment | LongCat-2.0 | 2.5 has no released weights; 2.0 is MIT on Hugging Face |
All figures sourced from the official LongCat-2.5 release blog (November 2026). Scores marked * originate from third-party reports or leaderboards; unmarked competitor scores were self-run by the LongCat team. This page is an independent summary and is not affiliated with Meituan.