AI security is a hot topic in the world of cybersecurity. If you don’t believe me, a brief glance at LinkedIn uncovers that everyone is an AI security expert now. This is why we end up with overly complex and sometimes nonsensical recommendations regarding the topic. But in the bustling market of thought leadership and job updates, we’ve seemed to have lost the plot. In most cases, it’s not AI security at all, but something else.

It’s Security From AI

I recently delivered the keynote at the c0c0n cybersecurity and hacking conference in India. It was truly an amazing experience. One of my takeaways was encouraging a shift in perspective on the term “AI Security,” highlighting how we often approach this topic from the wrong angle.

The term “AI Security” has become a misnomer in the age of generative AI. In most cases, we really mean securing the application or use case from the effects of adding AI. This makes sense because adding AI to a previously robust application makes it vulnerable.

In most cases, we really mean securing the application or use case from the effects of adding AI.

For most AI-powered applications, the AI component isn’t the end target, but a manipulation or entry point. This is especially true for things like agents. An attacker manipulates the AI component to achieve a goal, such as accessing sensitive data or triggering unintended outcomes. Consider this like social engineering a human as part of an attack. The human isn’t the end goal for the attacker. The goal is to get the human to act on the attacker’s behalf. Thinking this way transforms the AI feature into an actor in the environment rather than a traditional software component.

There are certainly exceptions, such as with products like ChatGPT, where guardrails prevent the return of certain types of content that an attacker may want to access. An attacker may seek to bypass these guardrails to return that content, making the model implementation itself the target. Alternatively, in another scenario, an attacker may want to poison the model to affect its outcomes or other applications that implement the poisoned model. Conditions like these exist, but are dwarfed in scale by the security from AI scenarios.

Once we start thinking this way, it makes a lot of sense. We shift to the mindset of protecting the application rather than focusing on the AI component.

AI Increases Attack Surface

Another thing to consider is that adding AI to an application increases the attack surface. Increase in attack surface manifests in two ways: first, functionally through the inclusion of the AI component itself. The AI component creates a manipulation and potential access point that an attacker can utilize to gain further access or create downstream negative impacts.

Second, current trendy AI approaches encourage poor security practices. Consider practices like combining data, such as integrating sensitive, non-sensitive, internal, and external data to create context for generative AI. This creates a new high-value target and is a poor practice that we’ve known from decades of information security guidance.

Also, we have trends where developers take user input, request code at runtime, and slap it into something like a Python exec(). This not only creates conditions ripe for remote code execution but also a trend where developers don’t know what code will execute at runtime.

Vulnerabilities caused by applying AI to applications don’t care whether we are an attacker or a defender. They affect applications equally. This runs from the AI-powered travel agent to our new fancy AI-powered SOC. Diamonds are forever, and AI vulns are for everyone.

It’s Simpler Than It Seems

Here’s a secret. In the real world, most AI security is just application and product security. AI models and functionality do nothing on their own. They must be put in an application and utilized in a use case, where risks materialize. It’s not like AI came along and suddenly made things like access control and isolation irrelevant. Instead, controls like these became more important than ever, providing critical control over unintended consequences. Oddly enough, we seem to relearn this lesson with every new emerging technology.

In the real world, most AI security is just application and product security.

The downside is that without these programs in place, organizations will accelerate vulnerabilities into production. Not only will they increase their vulnerabilities, but they’ll be less able to address them properly when vulnerabilities are identified. Trust me, this isn’t the increase in velocity we’re looking for.

I’ve been disappointed at much of the AI security guidance, which seems to disregard things like risk and likelihood of attack in favor of overly complex steps and unrealistic guidance. We security professionals aren’t doing ourselves any favors with this stuff. We should be working to simplify, but instead, we are making things more complex.

It can seem counterintuitive to assume that something a developer purposefully implements into an application is a threat, but that’s exactly what we need to do. When designing applications, we need to consider the AI components as potential malicious actors or, at the very least, error-prone actors. Thinking this way shifts the perspective for defending applications towards architectural controls and mitigations rather than relying on detecting and preventing specific attacks. So much focus right now is on detection and prevention of prompt injection, and it isn’t getting us anywhere, and apps are still getting owned.

I’m not saying detection and prevention don’t play a role in the security strategy. I’m saying they shouldn’t be relied upon. We make different design choices when we assume our application can be compromised or can malfunction. There are also conversations about whether security vulnerabilities in AI applications are features or bugs, allowing them to persist in systems. While the battle rages on, applications remain vulnerable. We need to protect ourselves.

There is no silver bullet, and even doing the right things sometimes isn’t enough to avoid negative impacts. But if we want to deploy generative AI-based applications as securely as possible, then we must defend them as though they can be exploited. We can dance like nobody is watching, but people will discover our vulnerabilities. Defend accordingly.