Transformers

High-Order Flow Matching: Unified Framework and Sharp Statistical Rates

In-Context Deep Learning via Transformer Models

This paper investigates the transformer’s capability for in-context learning (ICL) to simulate the training process of deep models, providing a provable explicit construction.

Computational limits of low-rank adaptation (lora) for transformer-based models

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models

We study the computational limits of Low-Rank Adaptation (LoRA) update for finetuning transformer-based models using fine-grained complexity theory.