AIQB
TutorialsOrdinary

Was TinyStories the Domain or the Vocabulary?

Source: DEV Community·

Summary

TinyStories is famous for a ~1,500-word vocabulary, so I capped mine at 1,500 and scored every arm on identical positions. It came out 4 points worse at top-1 than a 4,000-word cap, and more than a third of its output was the unknown-word token. The transferable lever was the domain, not the vocabulary.
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-be1121d327d708118b5dfaea