A New Way to Measure AI Safety for Kids

The rapid integration of AI into the lives of children necessitates standardized safety benchmarks to measure and improve model behavior. Mathilde Collin, founder of the software company Front, developed Kora, an open-source benchmark that evaluates AI safety across 26 risk categories. By utilizing LLM-based scenarios and human review, Kora identifies critical safety gaps, particularly in emotional support and educational applications where AI models often lack necessary friction. Data reveals that models perform significantly better when explicitly prompted to recognize a child user, highlighting the importance of age verification. Beyond technological safeguards, fostering AI literacy and encouraging traditional child development practices—such as independent play and human-centric social interaction—remain essential. Ultimately, moving beyond engagement-based metrics toward outcome-focused AI development is vital to ensuring that AI tools support, rather than replace, healthy human growth.