-
Matryoshka Language Model Suites in a thread
Social media posts for Matryoshka Language Model Suites.
-
Editing the methods of
sub-models are made of blocks
-
critique of matryoshka paper (GA26)
The narative about saving on parameters is diluted by a secondary point about cheap distiliation.
-
Week 0 in one page: tokens keep knowledge apart
The paper, what we learned, and everything the class wrote about it, in a five-minute read.
-
Three things to remember, and where the paper hides them
The points most readers should leave with, and how the paper could make them harder to miss.
-
The paper in four feeds
Posts for X, Bluesky, Mastodon and LinkedIn, and why each one is written differently.
-
The core idea in one picture
Same sentence, same model, different token IDs, and the knowledge stops flowing. A diagram of the three setups.
-
Rewriting the abstract: lead with the surprise
The paper's most striking result is buried in sentence six. Here is a version that starts with it.
-
Reimplementing CLeq: what does 12.6% actually mean?
I simulated the paper's measurement with a known amount of transfer. The metric is right at 0% and 100%, and generous in between.
-
Where this paper comes from: a literature map
The four lines of work it builds on, and the one it argues with.
-
Five follow-ups worth doing
Concrete next experiments, from a weekend project to a thesis chapter, and why each is worth the time.
-
Fictive facts: how to know exactly what a model saw
A five-minute lesson on the trick that makes controlled knowledge experiments in pretraining possible.