Clover-35B-A3B on llama.cpp: what "AR mode only" means, and a GGUF that answers in Gs

Running local LLMs via Ollama, LM Studio, or llama.cpp usually starts with the same guessing game:...

A local LLM chat app built on llama.cpp, Node and a hand-built web UI, plus what it took to make localhost secure and ship the same model on Android.

Every LLM you've used, ChatGPT, Claude, Gemini, Llama, does the same thing under the hood. It...

How I wrapped llama.cpp in a small, coroutine-first Kotlin API, and the three problems that took the most time: threads, cancellation and broken emoji.

I started working on Llama Legends, a browser-based card collector, in mid-December 2025. The goal...

A while back I benchmarked llama.cpp on a 96GB GPU setup and found its concurrency behavior...

A 74B vision agent model in 22 GB on one A40: 136 tokens/s with its drafter, 70 without, about 27 GB used. What the 24 GB leaves out.

Here is the exact exchange. The model is llama3.2:latest (3.2B, Q4_K_M), running locally through...

A week of benchmarking Muse Glimmer 30B on AMD Strix Halo with llama.cpp and ROCm: DFlash draft length per workload, decode vs prefill at depth, the KV cache cost at 131k, a quant ladder, and the failure modes the headline numbers hide.

Llama.cpp is a high-performance inference platform designed for Large Language Models (LLMs) like...