TutorialsOrdinary
Was TinyStories the Domain or the Vocabulary?
Summary
TinyStories is famous for a ~1,500-word vocabulary, so I capped mine at 1,500 and scored every arm on identical positions. It came out 4 points worse at top-1 than a 4,000-word cap, and more than a third of its output was the unknown-word token. The transferable lever was the domain, not the vocabulary.
CategoryAI Tutorials & Practice
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-be1121d327d708118b5dfaea