AIQB
TutorialsOrdinary

Running a dense 65B coder on one GPU: what memory bandwidth and 2-bit files cost you

Source: DEV Community·

Summary

Ripple-65B at 4 bits wrote 15 tokens/s on a 48 GB RTX A6000. Why dense caps speed, why long prompts stay cheap, and what 2-bit breaks.
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-3a5f43b920a85af1c801169f