There is a lot of noise about AI and security, much of it selling something. It is worth separating what genuinely changed from what is the same problem with better grammar.
What genuinely changed
The economics. Producing a convincing, personalised, well-written approach used to cost attacker time, which limited volume. It now costs almost nothing, which removes the limit.
That has three practical consequences.
- Phishing is fluent. Every awareness programme built on spotting poor spelling and odd phrasing is now training staff on a signal that no longer exists.
- Personalisation is cheap. Public information about your company, your suppliers and your staff can be aggregated into a plausible pretext in minutes.
- Voice and video impersonation is accessible. A phone call confirming a payment instruction is no longer sufficient verification on its own.
What did not change
The way in. Attackers still arrive through unpatched software, reused credentials and staff acting on a convincing instruction.
This matters because it means the defence is not exotic. Patch discipline, unique credentials with MFA, least-privilege access and tested recovery still address the overwhelming majority of real incidents.
Anyone selling an AI-specific security product before you have those in place is selling the interesting layer over the missing foundation.
If you have unpatched dependencies and shared admin logins, AI-era threats are not your problem yet. The ordinary ones are.
The genuinely new exposure: your own AI use
The most common new risk in businesses we audit is not an attacker. It is staff pasting confidential material into consumer AI tools to get work done faster.
Client data, contracts, unreleased pricing, source code and personal information all get pasted into whatever tool is open, usually with good intentions and no policy telling them otherwise.
The fix is a one-page policy naming what may and may not go into external tools, plus an approved tool with appropriate terms so people are not forced to improvise. Prohibition without an alternative is a policy people route around.
Replace detection with verification
Training staff to detect a sophisticated fake is a losing position and getting worse.
Verification procedures are not. A rule that any payment change or credential request is confirmed on a known number, initiated by the recipient, does not depend on anyone spotting anything.
Detection versus verification
One is getting harder every quarter. The other is not.
Detection — increasingly unreliable
- Spot the spelling mistakes
- Notice the odd phrasing
- Check whether the logo looks right
- Trust that the voice sounds familiar
- Judge whether the request feels plausible
Verification — still reliable
- Call back on a number you already held
- Two-person approval above a threshold
- Payment changes confirmed out of band
- Credential requests never actioned by message
- A stated policy that no urgency overrides
A proportionate response
For most businesses this is a short list, and none of it is exotic.
Update the awareness training to drop the grammar cues and teach the verification procedure instead. Write the AI usage policy and name an approved tool. Confirm patching and access control are actually disciplined rather than assumed. Test a restore.
That covers the realistic threat model for a business whose main asset is its acquisition channel. Anything beyond it should be justified by a specific risk you can name.