AIQB
TutorialsOrdinary

Muse Glimmer 30B on Strix Halo: a week of measurements

Source: DEV Community·
Llama

Summary

A week of benchmarking Muse Glimmer 30B on AMD Strix Halo with llama.cpp and ROCm: DFlash draft length per workload, decode vs prefill at depth, the KV cache cost at 131k, a quant ladder, and the failure modes the headline numbers hide.
CategoryAI Tutorials & Practice
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-1003509e62559c332af77999
Read original source ↗More Tutorials

Related stories

Matched by category, entity tags and source

TutorialsOrdinary
Building a Llama.cpp Container Image
Llama.cpp is a high-performance inference platform designed for Large Language Models (LLMs) like...
Source: DEV Community · Llama
TutorialsOrdinary
SlothTidy: A Zero-Friction, 100% Local AI File Daemon That Organizes Your Downloads While You Rest 🦥
Stop dragging PDFs and screenshots out of ~/Downloads. SlothTidy uses small open-weight models via Ollama and llama.cpp to automatically classify, rename, and organize your files 100% offline.
Source: DEV Community · Llama / Ollama
TutorialsOrdinary
PromptShield AI: Zero-Leak Local Redaction & Prompt Hardening Engine for Open-Weight Models
A local-first, zero-telemetry PII and secrets redaction proxy with bidirectional rehydration for open-weight models (Gemma, Llama, Mistral).
Source: DEV Community · Llama / Mistral
TutorialsOrdinary
vLLM vs. Ollama vs. llama.cpp: The Definitive LLM Inference Engine Guide
ENTERPRISE AI ARCHITECTURE & INFERENCE SYSTEMS ENGINEERING Technical Audit: PagedAttention KV...
Source: DEV Community · Llama / Ollama
TutorialsOrdinary
Meta AI training: what the new business terms ask of your website
Meta AI training appears explicitly in upcoming commercial terms. Here is who is covered, what the website permission says, and where robots.txt fits.
Source: DEV Community · Llama
TutorialsOrdinary
Local image generation with llamadart for the Llama Village storybook
Beyond chat with llamadart · Part 4 Dash missed it. Pip’s scarf is by the pond, and all he has is...
Source: DEV Community · Llama
TutorialsOrdinary
Local speech with llamadart: giving Llama Village a voice
Beyond chat with llamadart · Part 3 Pip’s scarf is missing. I wanted Dash to be able to fly over to...
Source: DEV Community · Llama
TutorialsOrdinary
Local embeddings with llamadart: how llamas share secrets
Beyond chat with llamadart · Part 2 Llama Village, where five llamas share a world without sharing...
Source: DEV Community · Llama
TutorialsOrdinary
An AI optimization made llama2.c finish sooner and start streaming later
This is a submission for the Kaggle Benchmarking Challenge. I built a Kaggle benchmark to test...
Source: DEV Community · Llama
TutorialsOrdinary
Llama Village: a virtual world powered by local AI with llamadart
Llama Village in motion. The characters’ dialogue is generated by an on-device model. Watch the...
Source: DEV Community · Llama