Fortra: AI Red Teaming Risks in Live Banking Environments

Name: Fortra: AI Red Teaming Risks in Live Banking Environments
Uploaded: 2026-08-23T09:28:46-04:00
Duration: 1 min 13 s
Description: AI can accelerate security testing, but should it be trusted inside a live banking environment? Pieter Ceelen, co-founder of Fortra’s Outflank, explains why AI may be valuable in a controlled lab but requires clear limits when red teams test production...

Fortra

08/23/2026

0 (0%)

Report Like Favorite

TL;DR

AI should not be given unrestricted access to live banking databases during red team engagements due to the risk of hallucinations or unintended destructive commands.
Isolated test lab environments are far safer contexts for AI-powered security analysis, where broader autonomy can be granted without production risk.
Red teamers must establish clear rules of engagement with clients that explicitly define where and how AI tooling may be applied during live system testing.

Summary

In this short clip from The Art of Security podcast, Pieter Ceelen, co-founder of Fortra's Outflank, makes a pointed case for why AI-assisted red teaming must be constrained when operating against live production systems — particularly in financial services. Drawing on firsthand experience penetration testing core banking environments, Ceelen describes scenarios where arbitrary query execution on live databases was possible, and explains why handing that capability to an AI agent introduces unacceptable operational risk. A hallucination or unintended command — such as a rogue DELETE query on a core financial database — could cause irreversible damage. His position is clear: the industry cannot afford that exposure. The clip draws a practical distinction between two red teaming contexts. In vulnerability research conducted inside isolated test labs, AI can be given broad autonomy and analytical power with minimal risk. But when testing live production systems, red teamers must define explicit rules of engagement with the client — agreeing in advance on where AI tooling can and cannot be applied, and why. This framing positions responsible AI use in offensive security not as a technology limitation, but as a professional and contractual discipline.

Chapters

0:00 - Live Banking Database Risks
0:27 - Industry-Wide AI Boundaries
0:36 - AI in Test Labs vs. Production
0:51 - Rules of Engagement with Clients

Key Quotes

0:09 "I don't want to AI to run the queries there or hallucinate an answer there or by accident run a delete everything from like a core financial."
0:27 "And as an industry, we can't be there."
0:44 "You can give much more power to the AI. Where if you're testing on live production systems, you probably want to be a bit more careful on some ends."

FAQ

Why is AI considered risky in live banking red team engagements?

AI models can hallucinate or execute unintended actions. In a live banking environment, this could mean running a destructive query — such as deleting records from a core financial database — with no ability to reverse the damage.

Where is AI-assisted red teaming considered safe to use?

Ceelen recommends using AI freely in isolated test lab environments, where a dedicated machine can be spun up for analysis without any risk to production systems or real customer data.

Categories:

» Data Protection

Tags:

Show more Show less

TL;DR

Summary

Chapters

Key Quotes

FAQ

Bridging the SaaS Protection Gap: Preventing Data Loss and AI Missteps

The Automation Escalation: Discovering the AI-Driven Underground Revolution

Invisible Data: Understanding What You Can't Safeguard