I built a private, offline AI chat you can run with one command: npx unmuzzled

Every LLM you've used, ChatGPT, Claude, Gemini, Llama, does the same thing under the hood. It...

How I wrapped llama.cpp in a small, coroutine-first Kotlin API, and the three problems that took the most time: threads, cancellation and broken emoji.

I started working on Llama Legends, a browser-based card collector, in mid-December 2025. The goal...

A while back I benchmarked llama.cpp on a 96GB GPU setup and found its concurrency behavior...

A 74B vision agent model in 22 GB on one A40: 136 tokens/s with its drafter, 70 without, about 27 GB used. What the 24 GB leaves out.

Here is the exact exchange. The model is llama3.2:latest (3.2B, Q4_K_M), running locally through...

A week of benchmarking Muse Glimmer 30B on AMD Strix Halo with llama.cpp and ROCm: DFlash draft length per workload, decode vs prefill at depth, the KV cache cost at 131k, a quant ladder, and the failure modes the headline numbers hide.

Llama.cpp is a high-performance inference platform designed for Large Language Models (LLMs) like...

Stop dragging PDFs and screenshots out of ~/Downloads. SlothTidy uses small open-weight models via Ollama and llama.cpp to automatically classify, rename, and organize your files 100% offline.

A local-first, zero-telemetry PII and secrets redaction proxy with bidirectional rehydration for open-weight models (Gemma, Llama, Mistral).