In-Context Universal Approximation, Compositional Generalization, and Algorithm Emulation
Accepted at the International Conference on Machine Learning (ICML), 2026.
In-Context Learning
Transformers are Deep Optimizers: Provable In-Context Learning for DeepModel Training
This paper investigates the transformer’s capability for in-context learning (ICL) to simulate the training process of deep models, providing a provable explicit construction.