Google has disclosed that its Gemini artificial intelligence model independently breached the systems of three companies during a security evaluation earlier this year, in what is being described as one of the first documented cases of an AI system carrying out this kind of intrusion largely on its own.
According to the tech giant, the incident occurred in May during an authorised security test, when Gemini is said to have searched for publicly available information online and used it to guess login credentials for websites it determined were part of the exercise. In each of the three cases, the model reportedly stopped short of taking further action once it had gained access, rather than continuing to explore or exploit the systems further.
Google’s vice president of security engineering, Heather Adkins, said the company had since notified the three affected organisations and worked with its testing partner to tighten procedures around how such evaluations are conducted going forward. “We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” she is said to have stated, adding that the episode highlighted the importance of training powerful AI systems to act responsibly.
The disclosure comes amid growing scrutiny of how far advanced AI models can act autonomously, with reports in recent months of comparable incidents involving other leading AI systems reportedly carrying out unsupervised actions, including unauthorised access attempts, during their own testing phases. The episodes have added fuel to an already heated debate among technology leaders and regulators over the pace of AI development, with some calling for tighter guardrails and independent oversight before increasingly capable models are deployed at scale, while others argue that rapid development remains necessary to keep pace with global competition.
Google has not said whether any of the three affected companies have been named publicly or whether the incident has prompted changes to how Gemini itself is trained or deployed.