AI Scam Detection: Can AI Detect Fake News and Phishing?

In a 2024 study, researchers had Claude 3.5 Sonnet read through phishing emails and try to catch them. It caught 97% of them. Zero false alarms. Sounds like a solved problem. It isn't. Those same researchers also used AI to write phishing emails. They found that automation made a phishing campaign up to fifty times more profitable to run. So the honest picture in 2026 isn't "AI catches scams." It's an arms race. The tools scammers use and the tools that catch them often come from the same technology, and both sides keep getting better at roughly the same speed. 

What AI actually looks at 

A modern detection system doesn't just scan for a few suspicious words. It checks four things at once. 

First, the language itself. Word choice, sentence structure, urgency cues, the kind of grammar mistakes that made old phishing easy to spot. Today's AI-written scams mostly don't have those mistakes anymore. 

Second, links and technical details. Does the URL point to a newly registered domain? Does an attachment behave oddly? Does a website copy a real brand's login page down to the pixel? 

Third, sender behavior. Not just who the message claims to be from, but whether that account normally emails you. Did it just have a password reset? Is it suddenly asking for something it's never asked for before? 

Fourth, unusual requests checked against context. A message asking you to urgently wire money gets a lot more suspicious when it follows a pattern, say, an executive's account showing an impossible-travel login right before the message goes out. 

The strongest systems look at all four signals together instead of judging each one alone. A message can look fine on its own and still be part of an obviously coordinated attack once you connect it to what else that account has been doing. 

Fake news detection works differently, and less reliably 

Phishing detection has a fairly clean target. A link either goes to a fake login page or it doesn't. Fake news detection doesn't have that luxury. Academic systems built on models like BERT report strong accuracy, often above 90%, but that's on the specific datasets they were trained and tested on. Real-world studies looking across different platforms report a wider range, roughly 78% to 94%. And they all note the same weak spot: these systems struggle with satire, opinion pieces, and anything where context decides whether a claim is misleading. The words alone don't tell you that. A joke written like a news story and an actual fake news story can look almost identical to a model trained mostly on word patterns. 

Why detection keeps slipping, even as it improves 

Three problems keep this from ever being fully solved. 

The first is concept drift. A model trained on last year's scam patterns slowly gets worse as scammers change tactics. That's a big reason there's no reliable way to catch a brand-new phishing technique on day one. 

The second is the false positive problem. Sounds minor until you're the one affected. Block too aggressively, and a security team spends real time chasing alerts that turn out to be a colleague's normal email. That's exactly the kind of alert fatigue that makes people start ignoring warnings altogether. 

The third matters most if you're just an everyday user. AI-generated scam messages have gotten good enough to erase almost all the old warning signs. Bad grammar, generic greetings, a sender name that doesn't quite match, those used to be reliable red flags. Now a language model can write a fluent, personal message that mentions a real coworker, a real project, and a real deadline, built for one specific target in seconds. The stuff that used to let a careful person catch a scam by eye is exactly what AI writing tools are best at removing. 

Why you shouldn't take an AI verdict as the final word 

None of this means AI detection is useless. It catches a huge amount that a person scanning their inbox manually never would. But treating a green checkmark or a "looks safe" label as proof is the wrong lesson here. A detector's confidence score just reflects how closely something matches patterns it has seen before. It isn't some deeper certainty about truth or safety. A genuinely new scam technique, by definition, won't match anything the system has seen yet. 

So treat automated screening as a first filter, not a final answer. A link flagged as safe? Still hover over it and check the actual domain before clicking. An email that passed the spam filter but asks you to urgently transfer money or share a password? Still verify it through a separate channel, a phone call, or a message on a platform the sender doesn't control. The systems keep improving. So do the scams they're meant to catch. Trusting your judgment entirely to software staying ahead is exactly what security researchers keep warning against.