This paper investigates the transformer’s capability for in-context learning (ICL) to simulate the training process of deep models, providing a provable explicit construction.

GERM removes outliers to improve genomic model efficiency, enabling quantization and LoRA fine-tuning with minimal degradation.

We study the computational limits of Low-Rank Adaptation (LoRA) update for finetuning transformer-based models using fine-grained complexity theory.