Dynamic Modification at Inference Time

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs (Ziyue Li, Yang Li, Tianyi Zhou, 2026): POLAR (program-of-layers): Adaptive computation at inference time by predicting input-specific execution paths that dynamically skip or repeat layer modules.

In-Place Test-Time Training (Guhao Feng, Shengjie Luo, Kai Hua, Ge Zhang, Di He, Wenhao Huang, Tianle Cai, 2026): Final projection matrices can act as adaptable fast weights, integrating the stream into dynamic weight updates for next token prediction, improving long-context performance.