AI Everyday #24 - Faster and Faster!

This week, the world of AI is buzzing with exciting developments. A leaked Mistral model, MiQ, is showing remarkable performance that rivals GPT-4. Additionally, a groundbreaking open-source dataset called Dolma, containing 3 trillion tokens, is paving the way for training powerful language models. Several open-source models, such as Lava and Olmo, are pushing the limits, with some even surpassing closed-source models like Gemini Pro in specific benchmarks. New research is also shedding light on efficient tool usage for language models, while innovative methods for stable identity preservation in image and video generation are on the rise. Notably, the Eagle 7B model, which doesn't rely on transformers, is proving to be competitive with its linear scaling, hinting at its ability to manage extremely long contexts. The pace of innovation in this field is astonishing, with many more breakthroughs happening beyond this summary.