Google's AI Broke Into Three Real Companies. All It Needed Was a Weak Password.
On 19 September, Google confirmed an unusual incident that had happened four months earlier and had not been made public until now: its Gemini model autonomously broke into the systems of three real companies.
This was not a deliberate cyberattack.
It happened during a controlled capture-the-flag exercise run by Irregular, the Israeli AI security company. The model was given a seemingly simple task: retrieve information from the systems of a fictional company inside a test environment.
Except two things went wrong.
The model was not supposed to have internet access — but it did.
And the fictional company name chosen for the exercise happened to belong to a real company.
From there, everything moved quickly.
Gemini searched the web, identified systems it believed were part of the exercise environment, and got into them.
In all three cases it stopped the moment it understood the systems were real.
But what matters here is not that an AI managed to break into a system.
What matters is how it did it.
Not a zero-day. Not a sophisticated vulnerability. A password.
In one case, Gemini simply tried again and again to guess the password — until it worked.
In the two other cases it found credentials that had been accidentally committed to a public repository, and used them.
That is all.
No new exploit. No zero-day. No supply-chain attack.
Just two very old techniques:
Guess a password, or find one somebody left exposed.
And that is exactly why this incident matters.
The three companies were not special targets. They did nothing unusual. Their systems were simply reachable from the internet, and their credentials were weak or already exposed.
That description fits countless production servers today.
What actually changed
Password guessing has existed for decades.
The novelty is not the technique.
The novelty is who performs it.
A human attacker has to pick targets, gather information, try, fail, change direction and keep going.
An autonomous agent can do all of that by itself.
It searches public information, cross-references sources, identifies services, tries credentials, reads the responses, adjusts its next step and continues.
Without getting tired.
Without getting bored.
At machine speed.
In this case, nobody even asked it to attack.
It was given a legitimate task with boundaries that were not good enough — and three real companies were breached as a side effect.
And that may be the most important point of all:
If an agent can do this by accident, you can imagine how low the bar is for doing it on purpose.
Why a WAF or CDN will not necessarily help here
The moment you hear "AI attacked a server," the instinct is to think about a WAF, a CDN or a cloud firewall.
But in this case, none of them were in the path.
The agent did not "pass through the edge."
It found exposed systems on the internet and connected to them directly.
And that is where the important distinction lies.
An edge layer can only protect traffic that passes through it.
If your origin server has a public IP address, and some service is open directly to the internet, it can be reached without going through the CDN or the WAF at all.
In other words:
The only place that reliably sees every connection arriving at your server is the server itself.
Every other layer protects only the traffic that actually travels through it.
What would have stopped Gemini?
Here it is important to be precise.
For the password guessing — yes.
Password guessing only works if the attacker is given enough attempts.
nuDefend detects repeated authentication failures and automatically blocks the source address after a small number of tries.
The block happens at the kernel firewall — before the next attempt even reaches the service.
An agent that needs hundreds of attempts simply does not get them.
On two of our own ordinary production servers, within a single 24-hour window, we recorded:
144 failed SSH attempts from 25 different addresses on one server.
344 attempts on the second.
73 of those attempts targeted the root account directly.
All of them blocked automatically.
And these are not servers under a targeted attack.
This is simply the constant background noise of the internet.
The same kind of behaviour Gemini displayed in the exercise — only at global scale, all the time.
For the credentials exposed in a repository — no.
If someone stored a password, an API key or a token inside a public repository, that is a process failure.
No agent running on the server can retroactively fix a credential that has already leaked.
Anyone promising otherwise is selling an illusion.
You need to scan repositories, run secret scanning, rotate exposed credentials, and make sure they never enter the code in the first place.
And for Gemini's internet access — also no.
That was a sandbox and permissions problem on Google's side.
The model was given a communication capability it was never supposed to have.
No protective software running at the breached company can fix a permissions mistake inside someone else's AI environment.
In other words:
nuDefend would not have prevented all three cases.
It would have stopped one of them.
But precisely the kind that repeats itself again and again, day after day, on almost every public server on the internet.
This is exactly why the defence belongs on the server
nuDefend installs directly on the server and operates where the connection actually arrives.
It blocks known-malicious IP addresses right at the kernel firewall — both inbound and outbound — before the traffic reaches your application.
It detects and blocks repeated credential-guessing attempts.
Its signed threat list refreshes every 30 minutes, so an IP address that turned malicious today can be blocked on your machines today, with no manual action required.
nuDefend also shows which services are currently listening on your server, and how exposed each one really is.
That matters, because it is exactly what an autonomous agent does at the start of an attack:
first, it tries to understand what is open.
The difference is that you get to see it before it does.
Reports are generated locally and stay on your server.
But this story points to a bigger problem
The most interesting failure in the Google incident was not at the three breached companies.
It was at the AI.
There was an autonomous workload with outbound access — and that access was never supposed to exist.
Which brings the question back to us.
If you are running today:
a local inference server,
an AI agent,
an automation system,
a vector database,
or an agent connected to internal systems,
do you actually know where that process is allowed to connect?
Usually the answer is no.
We know which ports are open inbound.
We know which processes are running.
But it is far harder to answer:
who does each process actually talk to, outbound?
And that is precisely the direction of the next layer we are building into nuDefend.
Not just seeing outbound traffic.
But learning over time which destinations each workload normally communicates with — and then allowing only what it needs.
In other words:
Learn first. Allow only what belongs.
Not because AI is more "dangerous" than any other software.
But because an autonomous agent makes decisions on its own.
And once a process makes its own decisions, you cannot rely on it simply behaving the way you intended.
It needs boundaries.
In the meantime? The boring things still work.
Do not leave services open to the internet if you do not need to.
Do not use passwords that are easy to guess.
Do not store credentials in code.
And do not let anything try again and again until it succeeds.
AI may be changing the cybersecurity landscape.
But a great many breaches still start with the same old mistake:
We left the door open too long.
Sources: CNN Business · The Hacker News · ABC News