AIQB
TutorialsOrdinary

A weekend with TensorFold on a MacBook: the engine mattered, the quant did not

Source: DEV Community·

Summary

A 27B dense model on my MacBook decodes at about 26 tokens a second. That is fine for chat and...
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-9b82e551894e28e3883e5533