Does sharing KV cache ruin alignment and why do we apply forward KL-Divergence

2 papers similar to parts of the paper

🔭 Paper scouting · Recent papers nobody is talking about
on Matryoshka Language Model Suites
by Adam Gabet · October 23, 2026 · edit this page
  • To Cache or Not to Cache: Understanding and Mitigating Safety Incompatibility in Cross-Model KV Cache Reuse (Lin, Liu, Liu & Wang, 2026): This paper shows that using the KV cache from a base model in its instruct version makes it lose its safety training, meaning it stops refusing harmful requests (Llama-3-8B answers 6% -> 31%). That raises the question: does Matryoshka training keep the safety of the larger model when its sub-models share a KV cache?
  • Divergence controls entropy in distillation (Zucchet & Linderman, 2026): This paper shows that when distilling a student model from a teacher, the choice of forward or reverse KL controls the confidence of the student model: forward KL makes it less confident than the teacher, reverse KL more confident. This might explain the stability of Matryoshka training, since its forward-KL distillation keeps the small models from collapsing onto a live teacher that is still training.


Enjoy Reading This Article?

Here are some more articles you might like to read next:

  • Visualizing 🪆 Models
  • Visualizing the core idea of Matryoshka LM Suites
  • Critique of Matryoshka Language Model Suites
  • How autoregressive and diffusion LMs imitate each other, and why it is inefficient
  • Matryoshka Language Model Suites in a thread