LFM2 1.2B-Tool

Q4_K_M·1.2B params·GGUF

intelligence: see on Artificial Analysis →

checkpoint: LiquidAI/LFM2-1.2B-Tool-GGUF:Q4_K_M

All runs (10)

Hardware	Backend	Shape	Conc.	Gen tok/s ↓	TTFT	TPOT (ms)	Out tok	Total	VRAM Δ
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	codegen	1	522.1	11ms	1.9	659	1.26s	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	agent	1	504.5	11ms	1.9	438	850ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	chat	1	499.9	10ms	1.9	100	198ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	rag	1	475.8	26ms	1.9	101	222ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	codegen	1	465.3	20ms	2.1	633	1.37s	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	chat	1	459.9	13ms	2.1	100	206ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	agent	1	444.0	23ms	2.2	314	707ms	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	rag	1	423.6	37ms	2.1	98	246ms	0.000 GiB
GeForce RTX 5070 · 11.94 GiB	llama.cpp b9174 (cuda)	agent	4	220.3	290ms	3.9	435	1.98s	0.000 GiB
GeForce RTX 3090 · 24 GiB	llama.cpp 59778f0 (cuda)	agent	4	200.4	304ms	4.2	362	1.72s	0.000 GiB

Environment

GeForce RTX 3090 · 24 GiB

cpuAMD EPYC 7302P 16-Core Processor

gpuNVIDIA GeForce RTX 3090

archNVIDIA

vram24 GiB (system 64.0 GiB)

power200 W / 450 W max(44% cap)

backendllama.cpp 59778f0 (cuda)

serverlemonade unknown

osUbuntu 24.04 LTS

kernel6.17.13-7-pve

driver590.48.01

python3.12.3

containerizedtrue

runs/cell5

warmups2

endpoint/v1/chat/completions

streamingtrue

GeForce RTX 5070 · 11.94 GiB

cpuAMD Ryzen 9 7900 12-Core Processor

gpuNVIDIA GeForce RTX 5070

archNVIDIA

vram11.94 GiB (system 30.4 GiB)

power250 W / 300 W max(83% cap)

backendllama.cpp b9174 (cuda)

serverlemonade unknown

osCachyOS

kernel7.0.0-1-cachyos

driver595.58.03

python3.14.4

containerizedfalse

runs/cell5

warmups2

endpoint/v1/chat/completions

streamingtrue