From AI to Agentic Security
The adoption of new technologies and platform shifts has always introduced new vulnerabilities and attack vectors for malicious actors to exploit; this has been a consistent pattern throughout the history of modern enterprise technology.
In the late 1980s and 1990s, network and server security became the frontline, with firewalls and antivirus software from pioneers like McAfee, Trend Micro, Symantec, and Cisco representing the first wave of commercial cybersecurity tools. Endpoint security rose to prominence in the 2000s as the internet expanded and portable devices, such as laptops, began to leave the safety of the corporate perimeter, creating new exposure points. This shift paved the way for companies such as CrowdStrike and SentinelOne to emerge as leaders. The next era was cloud and SaaS security, with players like Wiz (recently acquired by Google for $32B in an all-cash transaction) establishing the foundation for cloud protection, and identity security vendors like Okta securing and managing digital identities.
We’ve now entered the age of AI. Yet, despite the hype, enterprise adoption of generative AI has been slower and less successful than anticipated. According to The GenAI Divide: State of AI in Business 2025 (MIT), 95% of generative AI implementations are falling short. While we can question the methodology and the numbers may vary, it’s fair to say that security is a decisive factor in the success or failure of these projects. Meanwhile, attackers have already begun targeting LLMs and other AI models directly.
In response to growing risks, a specialized AI security ecosystem has rapidly emerged over the past three years, focusing on protecting models and safeguarding data privacy. This field is already experiencing significant consolidation, with major acquisitions happening at a rapid pace. We’ve seen Protect AI join Palo Alto Networks, Prompt Security go to SentinelOne, and Aim Security acquired by Cato Networks. The recent acquisition of our former portfolio company, Lakera AI, by Checkpoint is another acquisition realized in that space. These strategic moves clearly show that established security companies view AI security as a critical and foundational component of their future security posture.
However, this article will not dive deeply into AI Security, as this landscape has been well-documented in other blogs and reports. Instead, we will shift focus to a new and emerging threat vector: one that arises not from the models themselves, but from how they are used; i.e., via agentic systems. Just as each past platform shift brought new challenges for security teams, Agentic AI will be no exception.
Why Do You Need a Different Approach to Secure AI Agents
Agents are probabilistic, not deterministic, which means that instead of always returning the same output, the aim is to interpret goals, plan multi-step actions, and call tools or APIs, leading to varied behaviour. This autonomy, if adopted at scale, magnifies risk; an exploited agent doesn’t just leak data, it can move across systems, take actions, and cause cascading failures.
Since agents are built around LLMs, they inherit all the baseline risks of AI security (e.g., prompt injection, training data poisoning, model theft, etc.). However, the difference is that when an agent is compromised, the damage compounds, introducing a new range of risks unique to poorly designed AI agents:
- Privilege Escalation and Unauthorized Actions: AI agents often get access to tools such as APIs and internal systems to perform their tasks. A key new risk is an attacker tricking an agent into misusing its privileges, using its legitimate access in an unintended and malicious way. For example, a malicious prompt could trick a financial agent into performing an unauthorized transaction or a procurement agent into ordering fraudulent goods; this can evolve into a more severe privilege escalation attack. An agent often gets broad permissions (e.g., access to all documents within a project) to be effective. Suppose an attacker can trick the agent into misusing its initial, legitimate access. In that case, they can then leverage that access to gain even higher-level permissions or move laterally across an environment to perform actions beyond its original scope, like accessing or exfiltrating data they shouldn’t have been able to. This classic cybersecurity risk is amplified by the autonomous nature of agents.
- Memory and Context Poisoning: While some LLM-based applications use short-term memory to recall conversation history, an AI agent’s memory is often a persistent, active knowledge base of facts and observations that informs its actions: this makes agents more vulnerable to context poisoning. An attacker can inject malicious data into this memory to corrupt the agent’s understanding or rules. Unlike a simple chatbot, where this might lead to an incorrect response, for an agent, a poisoned memory could cause it to perform a harmful, real-world action days or weeks later, making it a more dangerous and harder-to-detect threat.
- Goal Manipulation and Unpredictable Behaviour: The multi-step, multi-turn nature of AI agents makes them susceptible to goal manipulation. An attacker can subtly alter the agent’s objective over time, causing it to perform actions that are initially harmless but eventually lead to a malicious outcome. The agent may also be susceptible to “hallucinations” that cause it to choose the wrong tool or perform an unintended action. This unpredictability, combined with its ability to act, makes for a dangerous security cocktail.
- Multi-Agent System Vulnerabilities: When multiple AI agents interact with each other to complete a complex task, new risks emerge. A compromised agent could act as a “Trojan horse”, injecting malicious commands or data into the system and influencing the behaviour of other agents, which creates a cascading failure effect, where a single point of compromise can lead to a systemic security breach.
There are already real-world examples of these risks. As stated by a Senior Engineering Manager of AI/ML at a Fortune 10 company:
“Over the past years, we’ve seen phenomenal growth in AI agents – and with it, a surge of security issues. These range from API and service account risks to broader non-human identity management challenges.
What makes this space different is that traditional rule-based, semantic analysis is no longer enough. With agents, we need behavioural analysis: understanding how an agent is acting, and allowing or blocking actions dynamically based on that behaviour.. We also need context-aware access control – security solutions must understand the context in which an agent is trying to access a resource and make decisions in real time.
Overall, the problem has become significantly more complex as AI agents spread across technical and business functions. Right now, it feels like a black box with little transparency, and we have to be very cautious in how we manage agent access. That opaqueness is a new challenge – and one we’re only beginning to solve.”
Notable recent incidents and real-world examples include the following:
- In May 2025, cybersecurity researchers reported that attackers were exploiting Microsoft’s Copilot AI agents embedded in SharePoint. By crafting malicious queries, they bypassed traditional security controls to extract passwords and access restricted files. As a result, these interactions did not appear in standard activity logs, allowing attackers to exfiltrate data without triggering alerts. (gbhackers)
- In July 2025, Replit’s internal AI coding agent severely misinterpreted instructions. With full system access, it deleted the company’s production database and went on to generate over 4,000 fake user accounts filled with fabricated data. (The Economic Times)
- In August 2025, researchers disclosed a high-severity vulnerability in the Cursor IDE that enabled full remote-code-execution (RCE). By sending poisoned data via MCP, an attacker could cause Cursor to silently rewrite and execute attacker-controlled commands under the user’s privileges, opening the door to ransomware, data theft, and other post-compromise actions. The issue was responsibly disclosed and patched in Cursor 1.3. (AIM Security)
These incidents make it clear that traditional security assumptions no longer hold. Agentic systems do not just expand existing attack surfaces; they create entirely new ones, where misalignment, over-permissioning, and opaque decision-making can be as dangerous as outside adversaries. To meet this challenge, we need a new security playbook tailored to the realities of agentic systems.
Building the Foundations of Agentic Security
The urgency comes down to how quickly agentic workflows gain traction. Companies want to move fast and let their teams experiment with building agents and workflows, but at the same time, they need visibility into how many agents exist, what they’re doing, and how access is controlled. It’s an unavoidable trade-off: long-term adoption won’t scale without security, yet security rarely feels urgent until adoption is already underway. Still, agentic workflows are inevitable, and enterprises that choose to lean in must be prepared for the security complications that follow. For those organizations, the following domains represent the initial building blocks for securing the agent lifecycle:
- Discovery and Inventory: Agents can run for a few seconds or persist over longer periods, and in many cases, they can spin up sub-agents or change behaviour mid-run. Discovery and inventory, in this context, means maintaining a real-time catalogue of every agent instance, its current state, and the resources it interacts with. Security will start with continuous visibility into what is operating, what data it touches, and which tools it controls; without this, the attack surface grows invisibly.
- Identity and Authorization: Each agent needs its own machine identity, completely separate from human accounts. That identity should use short-lived credentials and make every action traceable back to a single agent, not a shared proxy. This part becomes crucial to revoke access cleanly or audit what an agent actually did once it’s in production.
- Permissions and Dynamic Least Privilege: Once an agent’s identity is established, it should begin with no access or only the minimum required for its task. Permissions can then be granted just-in-time and automatically revoked once the task is complete. To further reduce risk, agents should operate within sandboxed environments that strictly limit the tools, data, and system resources they can touch. Sandboxing, combined with dynamic privilege management, contains the blast radius if an agent is compromised or manipulated. The challenge is that most current controls are static and broad, while agents operate fluidly and unpredictably. Building systems that can grant, withdraw, and enforce privileges dynamically, without introducing significant friction, remains one of the most challenging and critical problems in agent security.
- Monitoring and Auditing: Monitoring agents go beyond API calls and network traffic. You must establish a baseline of normal behaviour by tracking reasoning traces, audit trails, and goal execution. Any deviation, such as unusual tool chains, excessive calls, or unexpected context processing, should trigger alerts and isolation, ideally detected even during the agent’s reasoning or call preparation stage.
- Testing and Red Teaming: Continuous testing is essential to ensure agents operate safely within defined boundaries. Penetration testing, simulations, and edge-case probing help uncover ways agents might fail or be manipulated. The goal is to deliberately push agents into unexpected states to expose vulnerabilities before they are exploited in production. However, this kind of testing brings trade-offs. Agent workflows often split into multiple sub-tasks, each with its own data, prompts, or external calls, which makes thorough testing more complex and time-consuming.
The Initial Formation of an Ecosystem
As enterprises rapidly adopt AI agents, a supporting ecosystem of security providers is starting to form. Recognizing the scale of this opportunity, many companies are already making bold moves to capture early market share. We have grouped these players into four categories, mapped out in Figure II.
Security Incumbents: Legacy cybersecurity vendors that already dominate core categories (e.g., Identity with Okta) are beginning to repackage their offerings to address agentic security use cases. Rather than building new products from the ground up, they are extending existing suites – leveraging established telemetry, identity integrations, and enterprise distribution – to add agent-aware features.
Non-Human Identity (NHI) Security: Vendors historically focused on machine identities (e.g., service accounts, CI/CD credentials, device certificates) are now expanding to treat agents as another form of non-human identity. Many have already pivoted toward agentic security by adding connectors, webhook checks (event-driven calls to a security service for validation), OIDC checks (token-based verification of an agent’s identity), and pipeline-level testing.
LLM & AI Models Security: Vendors that specialize in securing LLMs and ML models running inside enterprise environments are also expanding their scope to agentic security. Traditionally, these companies have focused on building guardrails at the data, model, and runtime layers: securing prompts, detecting extraction or memorization attempts, and enabling runtime policy enforcement with model-aware telemetry.
Pure-Play Agentic Security: This emerging category is composed of startups building solutions from the ground up, designed solely for agentic security use cases. Unlike incumbents or adjacent players expanding into this space, these companies are focused entirely on the unique challenges of autonomous agents. Their solutions emphasize runtime guardrails, memory sandboxing, policy enforcement, and multi-agent orchestration. Although still nascent, the category is developing rapidly, with specialized startups already addressing discrete components of agent security with a high degree of focus and intensity.
We could also have included application security players beginning to extend into this space, approaching agentic risks through the lens of software supply chain and dependency management. Their moves further underscore how competitive and crowded this market is becoming. This quote from the CIO & Global CISO of a large publicly traded fintech company captures well the state of the early agentic AI security ecosystem:
“I already know of roughly 15 companies tackling agentic AI security, some still in the seed stage. Their approaches vary – some are working on security at the design and development level, others are treating it more like endpoint detection and response. A number of non-human identity vendors are also pivoting into this space. But most efforts today focus narrowly on discoverability or prompt security, and no company can yet deliver even 60% of what’s needed for full protection. The field is still early, fragmented, and being built step by step.”
Agentic systems change the equation for enterprise security. The challenge is not just a larger attack surface, but the dense interconnectivity they introduce across telemetry, logs, audits, identity, and access. This interconnectivity is already forcing both vendors and enterprises to rethink their security approaches and adapt existing tools to a new class of risks.
Agents might seem simple today, but their workflows are already growing more complex with every iteration. The bet is that they’ll become central to how enterprises operate, and if that is true, security has to evolve alongside them; it cannot be bolted on later. Only the systems that adapt quickly will earn the trust needed to scale.
Agentic security is still in its early days, but we believe that as agents are more widely adopted, they will create a security nightmare if left unaddressed. Many practitioners already recognize the risks, yet most early vendors are still experimenting and learning what it will take to secure these systems. Given how critical these specialized AI security tools will be in the future, our team is committed to deepening our understanding of this space and actively meeting with founders who are building solutions.
For further information, please reach out to Taha Mubashir, Etienne Gauthier, Ayush Malhotra, or Yoran Beisher.