For years, cybersecurity has operated on a simple premise: humans attack machines, and humans defend them. That premise is changing as autonomous AI agents gain the ability to write and execute code, access systems, identify vulnerabilities, communicate with other agents and pursue objectives with limited human intervention.
A recent incident involving OpenAI’s AI agents offers an early warning of what this could mean for enterprise security. According to reports, hundreds of AI agents involved in cybersecurity testing breached intended controls, accessed external systems and collaborated through an unauthorised communication channel. Investigations into the Hugging Face incident found that roughly 700 agents were involved, demonstrating how autonomous systems can potentially coordinate activity at a scale and speed that would be difficult for human operators to replicate.
The significance of this incident goes beyond AI safety. It raises a critical cybersecurity question: what happens when the attacker itself operates at machine speed, while the defender still depends on human response cycles?
The speed gap is becoming a security gap
Traditional Security Operations Centers (SOCs) integrate SIEM platforms, threat intelligence, security analytics and human expertise. Alerts are generated; analysts investigate them, build context and then decide how to respond. Even with automation, high-risk decisions still depend heavily on human judgement.
That model is more difficult to sustain when an adversary can work 24/7, analyze its environment and change its behavior without waiting for instructions.An autonomous attacker could potentially scan systems, test credentials, identify vulnerabilities and change tactics when blocked. If multiple agents can collaborate, the scale increases further.
This is why the future SOC cannot simply be an existing SOC with an AI tool added to it. It needs to become increasingly AI-native, capable of detecting, investigating and containing threats at machine.
Consider an enterprise AI agent that normally accesses a CRM system but suddenly requests access to a database, creates an unfamiliar API connection and attempts to use credentials outside its normal workflow. Each event might appear relatively low-risk on its own. Together, they could indicate that the agent has been compromised or is behaving outside its intended purpose.
An AI-native SOC should be able to identify that pattern, investigate the activity and, when confidence is high, automatically restrict access, revoke credentials or isolate the affected environment while escalating the incident to a human analyst.This is the foundation of AI-to-AI defence: intelligent systems monitoring other intelligent systems and containing machine-speed attacks, while humans remain responsible for decisions with significant business or operational consequences.
AI agents are the new identity challenge
One of the most significant shifts enterprises need to prepare for is the emergence of AI agents as a new form of digital identity.
Organizations used to control access for employees, contractors, applications and devices. AI agents can now work across these environments, with permissions to access databases, cloud infrastructure, APIs, code repositories or customer information.
If an agent is compromised, then its legitimate permissions could be an attack path for an attacker to the organization. Least-privilege access becomes more important. AI agents should have a clear identity, limited permissions, limited tool access, auditability of activity, and access should be granted based on their authorized purpose.
Increasingly, security teams will have to ask not only “Who accessed this system?” but “Why did this AI agent access it, what was it trying to accomplish and was that behavior consistent with its role?”
The defender can also become a target
There is another complication: the AI being used for defence can itself become part of the attack surface.
Security teams are increasingly using AI for threat hunting, investigation and automated response. Attackers could attempt to manipulate the data feeding these systems, exploit connected tools, abuse permissions or influence an AI agent through malicious instructions.
AI chat phishing introduces another challenge. As employees increasingly use AI platforms, attackers can exploit trusted AI interfaces to deliver convincing social-engineering and credential-harvesting attacks. For SOC teams, this can create visibility blindspots because encrypted LLM sessions can be difficult to inspect, while malicious activity may appear in network logs as traffic to a trusted AI domain or browser process. Automated campaigns can also increase the volume of alerts originating from seemingly legitimate internal AI usage.
This makes behavioural context and correlation increasingly important, as the AI interface itself can become part of the attack chain.
AI security therefore needs to extend beyond the model itself. Enterprises will require controls around agent identity, permissions, tool access, data sources and decision-making, alongside clear rules defining which actions an AI system can take independently and which require human approval.
Every autonomous security agent should also have an emergency stop mechanism. If its behaviour moves outside authorised boundaries, security teams must be able to immediately suspend its access.
Why Indian enterprises should prepare now
For Indian businesses, this is no longer a distant scenario. Banks, fintechs, healthcare organizations, manufacturers, telecom operators and digital commerce businesses all embed AI into customer service, software development, fraud detection, operations and decision making. Meanwhile enterprise environments are getting more interconnected through cloud platforms, APIs, third party applications and operational technology.
This creates a larger attack surface and increases the potential impact of a compromised machine identity.
Consider a manufacturing enterprise with an AI agent that can tap into analytics on production, enterprise applications and cloud systems. If that agent is compromised, the attacker may not need to initiate a traditional endpoint attack. It might have existing credentials or trusted API connections that give it a path to more sensitive systems.
It’s not just “malicious AI” that’s the risk. It’s also legitimate AI capabilities being repurposed for unintended results.
Building the AI-native SOC
The response requires security operations to evolve on several fronts, starting with continuous behavioral monitoring to establish what AI agents normally do and identify meaningful deviations. Identity, endpoint, cloud, network and application signals must also be correlated to identify attack chains rather than isolated anomalies.
High-confidence threats should trigger automated containment actions such as credential revocation, connection blocking, endpoint isolation or suspension of an AI agent. At the same time, every AI agent should have a defined owner, purpose, permissions, approved data access and escalation policy.
Human oversight will remain critical, particularly when decisions involve critical infrastructure, sensitive information or significant business disruption. AI should accelerate detection and response, but it should not remove human accountability.
Modern SOCs will therefore need to bring together SIEM, SOAR, UEBA, threat intelligence and AI-driven analytics to continuously identify, correlate and respond to threats rather than waiting for humans to connect the dots.
The next cybersecurity arms race
Cybersecurity has always been an arms race between offensive and defensive technologies. AI is now compressing the time available to respond, making security architecture designed for human-speed attacks increasingly inadequate for machine-speed threats.
The OpenAI incident is a warning of what this shift could mean. As autonomous systems become capable of circumventing controls, communicating with one another and exploiting weaknesses, the SOC cannot simply wait for alerts and investigate them one by one. It will need intelligent systems that continuously monitor other intelligent systems, recognise abnormal behaviour, understand attack paths and contain threats in real time.
The future of cybersecurity may therefore not be human versus machine. It could increasingly be machine versus machine—with humans deciding which side has the advantage.
Authored by Manpreet Singh, Co-Founder & Principal Consultant, 5Tattva
