TL;DR: We ran a human subject study on whether language models can successfully spear-phish people. We use AI agents built from GPT-4o and Claude 3.5 Sonnet to search the web for available information on a target and use this for highly personalized phishing messages. We achieved a click-through rate of above 50% for our AI-generated phishing emails.

Full paper: https://arxiv.org/abs/2412.00586

This post is intended to be a brief summary of the main findings, these are some key insights we gained:

AI spear-phishing is highly effective, receiving a click-through rate of more than 50%, significantly outperforming our control group.
AI-spear phishing is also highly cost-efficient, reducing costs by up to 50 times compared to manual attacks.
AI models are highly capable of gathering open-source intelligence. They produce accurate and useful profiles for 88% of targets. Only 4% of the generated profiles contained inaccurate information.
Safety guardrails are not a [...]