Skip to content

Benchmarks

Local LLM speed results across models, backends, hardware, and power profiles. Decode tok/s is the headline metric; latency, raw engine runs, and workload context stay visible in their own views.

1181 source rows835 matching source rowslatest run May 21, 2026schemas v1-v4source content/benchmarks/runs/
Filters
Advanced filters

Full row-level explorer. This is the place for raw shapes, hardware probes, cache/ppfix rows, dense power caps, and reruns.

27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2codegen1
63.7
335ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2agent1
61.7
525ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2chat1
59.5
259ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=3codegen1
59.2
372ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=3chat1
58.7
259ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=3agent1
57.8
537ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2rag1
55.9
879ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=3rag1
54.6
934ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2codegen1
53.2
317ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wcodegen1
50.6
380ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wchat1
50.5
265ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx1k_answer1
50.2
1.48s0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wagent1
49.9
623ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2chat1
49.6
258ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-2-pl-200wcodegen1
48.7
390ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2rag1
47.8
854ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx1k_answer1
47.5
1.45s0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-2-pl-200wchat1
47.4
275ms0.0———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-2-pl-200wagent1
47.2
646ms0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wrag1
45.1
1.05s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 + cachectx32k_agent_cache1
44.2
1.18s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinechat1
42.7
238ms23.4———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-2-pl-200wrag1
42.5
1.13s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinerag1
42.3
738ms23.6———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinecodegen1
42.2
355ms23.7———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselineagent1
42.1
505ms23.8———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselineagent4
41.9
19.73s23.8———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx1k_probe1
40.1
1.09s24.9———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx1k_answer1
39.5
1.18s25.3———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx4k_answer1
39.1
3.05s25.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 + cachectx64k_agent_cache1
38.9
1.64s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx4k_answer1
38.0
4.57s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx16k_probe1
37.8
11.21s26.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx16k_answer1
37.6
11.27s26.6———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx4k_answer1
36.7
4.59s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx4k_probe1
36.5
3.05s27.4———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx32k_probe1
36.4
23.72s27.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx32k_probe1
36.1
23.85s27.7———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline + cachectx32k_agent_cache1
36.0
965ms27.8———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx32k_answer1
35.8
23.96s27.9———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx32k_answer1
35.7
23.87s28.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiBcap 200 Wdrv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wchat1
34.2
283ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 + cachectx128k_agent_cache1
33.9
2.58s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx64k_probe1
33.2
54.77s30.2———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx64k_answer1
33.0
54.96s30.3———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx64k_answer1
32.8
54.68s30.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline + cachectx64k_agent_cache1
32.7
1.24s30.6———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx64k_probe1
32.0
55.14s31.2———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiBcap 200 Wdrv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wrag1
31.2
1.05s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiBcap 200 Wdrv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wagent1
31.2
621ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiBcap 200 Wdrv 590
llama.cpp 4f13cb7-mtp (cuda)mtp-3-pl-200wcodegen1
31.1
384ms0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx100k_probe1
29.9
100.77s33.4———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx100k_answer1
29.1
100.91s34.4———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinechat1
28.1
236ms35.6———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinecodegen1
28.0
328ms35.7———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselineagent1
28.0
516ms35.7———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselinerag1
28.0
810ms35.7———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB each450 W × 2 maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)baselineagent4
28.0
28.39s35.7———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline + cachectx128k_agent_cache1
27.4
1.91s36.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx128k_answer1
27.4
142.02s36.5———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baselinectx128k_probe1
27.2
141.38s36.8———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx128k_answer1
27.1
142.04s36.9———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)baseline-pl-200wchat1
27.1
238ms37.0———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)baseline-pl-200wcodegen1
26.9
337ms37.2———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)baseline-pl-200wrag1
26.9
911ms37.2———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)baseline-pl-200wagent1
26.8
625ms37.4———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)baseline after prompt fixctx128k_probe1
26.4
141.98s37.9———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=2agent4
25.3
12.42s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
GeForce RTX 3090 · 24 GiB450 W maxdrv 590
llama.cpp cuda-4f13cb7 (cuda)MTP n=3agent4
24.0
13.43s0.1———
27B-MTPthinkQ8_0legacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 590
llama.cpp 4f13cb7-mtp (cuda)baseline-pl-200wagent4
18.2
3.11s55.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx16k_answer1
17.2
19.15s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx16k_answer1
17.2
19.18s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx32k_answer1
10.3
38.50s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx32k_answer1
9.9
41.12s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx32k_answer1
9.8
40.99s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx1k_probe1
5.4
1.37s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx1k_probe1
5.3
1.37s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx64k_answer1
5.0
90.32s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx64k_answer1
4.7
95.98s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx64k_answer1
4.7
95.67s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx100k_answer1
2.7
176.47s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx100k_answer1
2.7
176.82s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx128k_answer1
2.0
242.34s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx128k_answer1
1.9
252.98s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx128k_answer1
1.9
254.00s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx4k_probe1
1.7
4.60s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx4k_probe1
1.7
4.67s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx16k_probe1
0.4
19.04s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx16k_probe1
0.4
19.08s0.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx32k_probe1
0.2
38.40s0.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx32k_probe1
0.2
40.97s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx32k_probe1
0.2
41.01s0.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx64k_probe1
0.1
90.40s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx64k_probe1
0.1
95.39s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx64k_probe1
0.1
95.65s0.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx100k_probe1
0.0
176.03s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx100k_probe1
0.0
176.45s0.0———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3 after prompt fixctx128k_probe1
0.0
242.16s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=2ctx128k_probe1
0.0
253.42s0.1———
27B-MTPthinkQ4_K_Mlegacystack comparable
2× GeForce RTX 3090 · 24 GiB eachcap 200 W × 2drv 595
llama.cpp direct (cuda)MTP n=3ctx128k_probe1
0.0
252.82s0.0———
Decode tok/s
Headline speed metric
TTFT / TPOT
Latency context
Raw vs workload
Separate comparison contracts
Notes badge key
hardware comparable

Use these rows for GPU-to-GPU comparisons when the model, quant, backend, driver family, power policy, and benchmark shape match closely.

stack comparable

Use these rows to compare a similar software stack. They are useful, but backend, server path, driver, cache, or power settings may still influence the number.

stack realistic

Treat these as real workload measurements, not pure hardware rankings. They include prompt mix, API/server overhead, cache behavior, and local software details.

legacyOlder workload harness row.
350 W capRecorded GPU power limit.
drv 590GPU driver branch.
reasoningReasoning-token model.
Metric guide
Decode tok/s - Generation rate. Raw rows come from the engine benchmark; API rows use token intervals when available.
TTFT - Time to first token. This includes prompt processing and server/API overhead.
TPOT / ITL - Time per output token after the first token. Lower is better.
Raw Engine - llama-bench style cases intended for hardware-normalized comparison across rigs.
Workload / API - Stack-realistic measurements that include backend, server, cache, driver, and prompt behavior.
Power badges - A cap badge shows the recorded power limit. The row metadata records the cap relative to the recorded max.