On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention

Xiaol.x - On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention