DiscoGen: Learning to Discover Learning Algorithms

DiscoGen addresses the limitations of current machine learning algorithm discovery benchmarks by providing a procedural generation framework capable of creating over 100 billion unique tasks. By strictly decoupling meta-training and meta-testing data, this system prevents contamination and enables the development of more robust, generalizable algorithms. The framework allows researchers to optimize for diverse objectives, including energy efficiency, training speed, and code length, rather than focusing solely on performance. Experimental results demonstrate that increasing the number of tasks during the meta-optimization process leads to improved performance on held-out domains, validating the effectiveness of large-scale meta-meta-learning. This approach shifts the research paradigm from static, limited benchmarks toward dynamic, scalable environments, offering a viable path to discover more efficient and versatile machine learning algorithms that perform reliably across varied, unseen problem spaces.