Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models

Xiaol.x - Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models