Building an open AI company - with Ce and Vipul of Together AI

Together AI aims to democratize AI development by building an open, independent, and decentralized platform for model training and inference. Moving beyond the limitations of hyperscalers, the company utilizes a disaggregated cloud of AI supercomputers to provide efficient, accessible infrastructure. Central to this mission is the Red Pajama project, which offers open-source datasets and modular quality-filtering tools to improve model performance. By jointly optimizing algorithms, hardware, and data, the company addresses the diminishing returns of current transformer architectures. Significant research efforts are dedicated to state-space models and hybrid architectures, which promise greater efficiency and long-context capabilities. This full-stack approach, combined with a commitment to open-source transparency, seeks to empower developers to build custom models without the high capital barriers typically associated with frontier model labs.