Benchmarks
Local LLM speed results across models, backends, hardware, and power profiles. Decode tok/s is the headline metric; latency, raw engine runs, and workload context stay visible in their own views.
content/benchmarks/runs/Advanced filters
Hardware tested
Rig metadata and microbenchmarks are shown here so memory bandwidth and tensor math do not get mixed into model-serving rankings.
| cap | theory | copy | fp16 | bf16 |
|---|---|---|---|---|
| fixed | 256 GB/s | 106 GB/s | 30.3 TF | - |
Framework Desktop with the AMD Ryzen AI Max+ 395 (Strix Halo) APU. 128 GiB of unified LPDDR5X system memory; the GPU side sees 96 GiB through the unified-memory pool. Integrated Radeon 8060S handles the inference workload via ROCm. No discrete GPU, no separate VRAM pool — the 27B-class models in this benchmark set all run on a single APU.
Best workload row per rig
| Rig | Best workload row | Decode tok/s | Backend / mode |
|---|---|---|---|
| Strix Halo · Radeon 8060S · 128 GiB unified (96 GiB VRAM) | LFM2 1.2B · chat | 211.5 | llama.cpp baseline |
Use these rows for GPU-to-GPU comparisons when the model, quant, backend, driver family, power policy, and benchmark shape match closely.
Use these rows to compare a similar software stack. They are useful, but backend, server path, driver, cache, or power settings may still influence the number.
Treat these as real workload measurements, not pure hardware rankings. They include prompt mix, API/server overhead, cache behavior, and local software details.