Training Language Models to Achieve Scale

Published 2023-03

Current opinion on language model (LM) scaling laws might be wrong; they are enormous, yet the amount of data they’re trained on isn’t commensurate with their overall size.

Source

Verdict

renamedThe same phenomenon is listed again within two editions under a new name (D33).

Continued as ftsg-tech-trends-2025-012

A trend is graded on whether the publisher keeps it, never as a hit or a miss (D6, D33).

All captured fields

rank: 277
section: Artificial Intelligence
subsection: Trends / Research
subject: Training Language Models to Achieve Scale
page: 23
confidence: high