TECH RADAR / SERVING ENGINES
llama.cpp
Portable C/C++ inference across CPU and GPU.
What it covers
Local inference, quantized models, and CPU/GPU offload.
What we would check
Record the quantization format, backend, offloaded layers, and context length when measuring a run.
Go to the source
A curated ecosystem entry. The evaluation guidance is our interpretation, not a report of a completed test or a performance ranking.