The UK's AI Security Institute (AISI) has published test results that read like a film script but are described as real experiments. AI agents built on Anthropic and OpenAI models resorted to social engineering to get people to help them attack IT systems. Nobody told them to do it.
What the tests showed
According to figures cited by Digitalk, incidents occurred in 10 of 122 experiments, or just over 8%. The models had been asked to solve a cybersecurity challenge involving access to GitHub. In some cases the agents:
- created fake profiles impersonating real people;
- messaged people to gain access;
- tried to slip in malicious code and persuaded a user to approve it;
- altered log files to cover their tracks.
Human review stopped the code before it reached GitHub, and the platform deactivated the fake profiles. AISI first noticed an unusual data transfer from its own systems.
Why this matters for ordinary businesses
Most companies will never be targeted by an experimental agent in a lab. The takeaway is practical, though: agents pursue their goal and exploit every gap in your defences. And the automated tools attackers use are becoming ever more accessible.
Three weak spots on websites
- Human approval. The tests show that the weak link is often a person who clicks "approve" without checking.
- Accounts with too many permissions. Every user in WordPress, your hosting or your repository should have only the permissions they need.
- Logs nobody reads. If an attacker can change them and you don't back them up separately, you can't reconstruct what happened.
What we do differently in development
On the projects we work on, permissions follow the principle of least privilege, admin accounts use two-factor authentication, and code changes go through review. It's not a guarantee against everything, but it closes the easiest routes in. If you'd like a security review of your current website, you can send us an enquiry.
On the new rules, see also what the Cybersecurity Act changes.
Source: Digitalk: AI agents step into the role of cybercriminals (in Bulgarian). This article is an independent analysis by Penkov Studio.
