The Future of Phishing
AI is increasing the scale and sophistication of phishing attacks, while dramatically lowering the cost of creating them. See it in action with a celebrity or fictional character.
Enter the name of a public or fictional person and then click the button to plan a phishing attack.
Email phishing—sending someone a deceptive email that tricks them into sharing sensitive information—is as old as the internet itself. Historically, this looked like either generic messages sent en masse, or highly personalized messages tailored to a small number of VIP targets. But due to advances in AI, creating targeted phishing attacks is easier, cheaper, and quicker than ever—meaning cybercriminals can treat everyone like a VIP.
Since AI systems can trawl through huge amounts of publicly available information, they can near-instantly learn who you are, what you're interested in, and what psychological vulnerabilities could be exploited. Using this information, they can then draft highly personalized emails that would appear authentic and trustworthy to an unsuspecting eye. This makes it much harder to tell what is legitimate and what is dangerous.
See for yourself, as AI writes a personalized email based on the information it compiled in the first demo:
No target has been selected. Please choose one above.
Notice how the generated emails vary not just in their content but in the strategy used to hook you. The fake name of the sender, as well as the tone and style of the email, vary per target. For example:
- Emails to a famous athlete may highlight a brand partnership relevant to their region
- Emails to a professor may contain an invitation to a niche conference in their field
- Emails to a business owner may present a salient partnership opportunity
Also notice that each email ends with an authentic-seeming link to the sender's LinkedIn profile. Clicking on this link takes you to a fake sign-in page inviting you to enter your LinkedIn password, which attackers will be able to see.
You can see how this could play out below:
Phishing at Scale
Phishing is by far the most common type of cybercrime, according to the FBI's Internet Crime Complaint Center. While it's been on the rise for years, data from Statista shows that, across the globe, the number of phishing attacks more than doubled in the year following ChatGPT's release. Data also suggests that 91% of all cyberattacks start with email.
AI is increasing not only the quality of cyber-attacks, but also the number of them. This will lead to a significant amplification in cybersecurity losses. Recent research has shown that it is already possible to fully automate the entire phishing process, and that today's best AI models perform on par with human experts at this task.
What Lies Ahead
Billions of people open emails every day. Previously, most of them were not worth phishing because the effort for criminals outweighed the payoff they could gain. But now, sophisticated scams can be crafted in minutes rather than hours, which changes the calculus and puts everyone at risk.
There is some evidence that AI can also be used to defend against phishing by proactively identifying ill-intent; but with AI systems improving monthly, attacks remain easier than defense.
CivAI is a 501(c)(3) non-profit that educates the public about AI's capabilities and dangers using live software demonstrations. Instead of research papers and abstract discussion, we produce interactive software experiences that provide a deep, intuitive sense of what AI can do and where it’s going. We present these demos to targeted audiences and make scalable versions available to the public.
To date, CivAI has delivered 100+ briefings to organizations across civil society (on AI-cyber risks, deepfakes, and elder fraud) and government (on biological risks, elections, and more). Our work has been featured on national television by ABC News and FOX, and in print by the New York Times, the Washington Post, TIME, CNN, and many others.
