LFM2.5-350M

Q4_K_M·350M params·GGUF

intelligence: see on Artificial Analysis →

checkpoint: LiquidAI/LFM2.5-350M-GGUF:Q4_K_M

All runs (10)

Hardware	Backend	Shape	Conc.	Gen tok/s ↓	TTFT	TPOT (ms)	Out tok	Total	VRAM Δ
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	codegen	1	861.4	8ms	1.1	672	763ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	agent	1	824.4	7ms	1.2	180	218ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	codegen	1	813.8	12ms	1.2	527	633ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	rag	1	792.4	13ms	1.1	64	81ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	chat	1	761.9	6ms	1.2	49	63ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	chat	1	722.7	9ms	1.2	41	57ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	agent	1	715.9	28ms	1.2	141	197ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	rag	1	632.0	23ms	1.2	59	93ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	agent	4	340.3	132ms	2.1	197	551ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	agent	4	316.9	206ms	2.1	186	617ms	0.000 GiB

Environment

GeForce RTX 3090 · 24 GiB

cpuAMD EPYC 7302P 16-Core Processor

gpuNVIDIA GeForce RTX 3090

archNVIDIA

vram24 GiB (system 64.0 GiB)

power200 W / 450 W max(44% cap)

backendllama.cpp 59778f0 (cuda)

serverlemonade unknown

osUbuntu 24.04 LTS

kernel6.17.13-7-pve

driver590.48.01

python3.12.3

containerizedtrue

runs/cell5

warmups2

endpoint/v1/chat/completions

streamingtrue

GeForce RTX 5070 · 11.94 GiB

cpuAMD Ryzen 9 7900 12-Core Processor

gpuNVIDIA GeForce RTX 5070

archNVIDIA

vram11.94 GiB (system 30.4 GiB)

power250 W / 300 W max(83% cap)

backendllama.cpp b9174 (cuda)

serverlemonade unknown

osCachyOS

kernel7.0.0-1-cachyos

driver595.58.03

python3.14.4

containerizedfalse

runs/cell5

warmups2

endpoint/v1/chat/completions

streamingtrue