AI Security Is Never Finished: Building the Continuous Red Teaming Loop 

Security programs are built around moments of closure: the finding is closed, the control passed, and the release can move forward. 

AI security refuses to cooperate with that model. 

A passing test is useful evidence, but only for a specific system, configuration, and moment. 

Production AI keeps moving. Models change behavior, prompts are revised, retrieval sources are added, agents gain tools, and users introduce unexpected context. Attackers adapt just as quickly. Yesterday’s clean result may not describe tomorrow’s risk. 

That has been the practical case for continuous AI red teaming. Now, research from the National Institute of Standards and Technology provides a mathematical basis for it. 

In Robust AI Security and Alignment: A Sisyphean Endeavor?, NIST senior scientist Apostol Vassilev applies the logic of Gödel’s incompleteness theorem to AI security. The conclusion is uncomfortable but useful: no finite set of guardrails can be universally robust against adversarial prompts. 

That does not make AI guardrails pointless. It makes permanent assurance the wrong objective. 

Why AI Security Has No Final Test 

AI guardrails are rules and controls designed to keep a system within policy. They may inspect inputs and outputs, restrict tools, filter content, enforce permissions, or monitor behavior. Each can make an AI system meaningfully safer. 

But every defensive system operates with limits. 

Human language gives attackers an enormous space in which to work. Harmful intent can be hidden through obfuscation, role-play, indirect instructions, gradual multi-turn escalation, unusual phrasing, another language, or context retrieved from an external source. For AI agents, the attack may also move through documents, email, websites, APIs, or connected tools. 

As the NIST explanation of the research puts it, the number of ways adversaries can hide harmful intent is effectively limitless. A fixed defensive model cannot anticipate every possible formulation. 

This is the distinction between defeating known attacks and proving that no unknown attack can succeed. 

Defenders can block a jailbreak, close an indirect prompt injection path, tighten a permission boundary, or stop an unsafe tool call. Those improvements matter. But passing all known tests does not prove that the system will withstand every prompt, context, or attack path it has not encountered yet. 

AI security therefore has no final exam. It has an improvement process. 

The Security Problem Is Permanence, Not Prevention 

The wrong lesson from the NIST research would be that prevention is futile. 

The more useful lesson is that prevention must keep learning. 

NIST’s proof does not give attackers a recipe for constructing the next successful prompt. Defenders can still remove known weaknesses and force adversaries to search for new ones. Every attack path that is discovered, understood, and closed increases the effort required to find another. 

That changes the measure of success. The goal is not to certify that an AI system is secure forever. It is to make successful exploitation progressively harder, more expensive, easier to detect, and less damaging when it occurs. 

This is familiar territory for security teams. No organization expects a firewall rule, detection signature, vulnerability scan, or incident response plan to remain sufficient forever. What is different about AI is the speed and ambiguity of the environment. Natural language is both the interface and part of the attack surface. The same request can be expressed in countless ways, and the system’s response can depend on context that was not present in the last test. 

NIST’s practical recommendation is direct: organizations must commit to a constant search for weaknesses and stay ahead of attackers. 

The Security Model NIST Points Toward 

The NIST model has three connected elements. 

The first is continuous adversarial discovery. Red teams actively search for prompts and attack paths that can push the system outside its intended boundaries before real attackers find them. That search must extend beyond obvious jailbreaks to include retrieved context, tool use, permissions, multi-turn behavior, and application-specific business logic. 

The second is continuous hardening. A finding should change something. Teams may revise a prompt, strengthen a policy, restrict a data source, narrow a permission, update a guardrail, redesign a workflow, or add a new runtime control. The discovered attack should then become part of the regression suite so the same weakness does not quietly return. 

The third is operational resilience. No prevention model justifies assuming that every attack will be stopped. Organizations also need to constrain what an AI system can affect, detect suspicious behavior, preserve evidence, limit the impact of a successful exploit, and recover quickly. 

These are not three separate projects. Discovery supplies evidence for hardening. Hardening improves production controls. Production monitoring reveals new behavior and attack patterns. Those observations create the next testing priorities. 

The result is a loop rather than a finish line. 

Red Teaming Creates the Feedback 

In AI Red Teaming Makes the Unknowns Known, we described how adversarial testing turns a broad concern into a specific finding: this system, in this configuration, can be manipulated through this path, causing this impact. 

At enterprise scale, that testing has to combine broad coverage with enough depth to understand how models, prompts, context, tools, permissions, and workflows interact. 

The next step is to make every useful finding part of the security program. 

A mature workflow does not end when a red team demonstrates a successful attack. It identifies the failed control and the business impact. It gives the system owner evidence to make a change. It re-runs the attack to verify the fix. It preserves the test so that future model, prompt, tool, or workflow changes can be checked against the same failure mode. 

That creates a practical operating loop: 

Discover -> Threat model -> Red team -> Prioritize -> Harden -> Protect -> Monitor -> Re-test 

Some testing should run on a schedule. Some should be triggered by change: a new model, a revised system prompt, an added retrieval source, a new connector, expanded agent permissions, or a changed workflow. Newly discovered attack techniques and production incidents should also feed directly into testing priorities. 

This is how threat intelligence becomes operational. A technique observed against one AI system can inform what defenders test across others. The result is not a static library of malicious prompts. It is a continuously updated understanding of how attacks evolve and how they propagate through real AI architectures. 

From Testing to Production Protection 

Red teaming shows what must be stopped. Runtime security provides the opportunity to stop it while the system is operating. 

If testing reveals sensitive data leakage, teams may need to change retrieval access, strengthen data controls, revise prompts, or inspect inputs and outputs at runtime. If an agent can misuse a permitted tool, the answer may involve narrower permissions, stronger approval boundaries, or controls that evaluate the action before it executes. If malicious instructions arrive through trusted content, defenses may need to inspect external context before it shapes the model’s behavior. 

This is why testing, monitoring, and protection cannot operate as isolated layers. 

The Check Point AI Defense Plane brings together discovery, protection, governance, and assurance across employees, AI applications, and agents. Assurance continuously tests whether systems and controls behave safely. Runtime protection acts where prompts, data, outputs, tool calls, and agent actions create production risk. Governance determines the policies and boundaries those controls enforce. Discovery shows where AI exists and what it can reach. 

Each capability makes the others sharper. Testing without production protection finds weaknesses but does not stop live exploitation. Protection without adversarial testing may defend against what teams already understand while missing system-specific attack paths. Monitoring without a remediation and re-testing loop produces activity rather than improvement. 

Continuous AI security depends on the connections between them. 

Securely Doing Business Requires a Living Program 

In Redefining the CISO Contract: From Securing the Business to Securely Doing Business, Adam Ely, GM of AI Security at Check Point, argues that security leaders should help their organizations move with AI rather than become the reason they cannot. 

The NIST findings reinforce what that requires operationally. 

Giving the business confidence to deploy AI cannot mean offering permanent approval based on a pre-launch result. Models are retrained, agents evolve, connections expand, and the risk profile of a deployment changes. Confidence has to come from knowing that the organization can see those systems, test them under adversarial pressure, protect them in production, and respond when new weaknesses emerge. 

That does not mean applying maximum testing to every AI use case. Risk should determine depth and frequency. An internal summarization tool and an agent with access to customer records and financial workflows should not receive identical treatment. 

It does mean treating assurance as a recurring security function for systems where failure could create material impact. Testing should begin early, continue around meaningful changes, and remain connected to engineering, governance, runtime controls, and incident response. 

That is how security becomes an enabler. It replaces a brittle choice between “move fast” and “stay safe” with an operating model designed to do both. 

Confidence Without Complacency 

NIST’s research does not tell organizations to surrender to inevitable AI failure. It tells them to abandon the comforting idea that one successful test or one carefully designed set of guardrails can settle the question forever. 

The stronger model is active. Search for weaknesses before attackers do. Turn findings into better defenses. Protect systems while they operate. Limit the impact of what gets through. Feed every lesson back into the next test. 

The question is no longer whether an AI system can be declared secure forever. It is whether the organization can discover weaknesses, strengthen defenses, and reduce impact faster than attackers can adapt. 

For a deeper look at why point-in-time testing creates false confidence, download the white paper: Why Your AI Passes Tests But Still Fails in Production. 

To learn how Check Point helps organizations continuously pressure-test AI systems and turn findings into actionable improvements, explore Check Point AI Red Teaming.