Transcript
65% of it is tied to identity-based tactics. So, social engineering, fishing, brute force. These are all identity-based tactics, and they show up every day. Just one recent example from a Unit 42 IR investigation that my team is working on right now, started with a fish. Fishing is very, very popular. That's not going away anytime soon. The threat actor landed on a typical user workstation. They used the tactic to sniff traffic, to grab a service account. That service account had domain admin. From there, they pivoted to the domain controller. They pivoted to the VMware ESXi server. They deleted the Active Directory environment. They blew away the ESXi environment, and they shut that company down. This is so common, we see it every day. Think about that's the way it works now, and think about how much faster this happens when frontier AI is in the hands of the adversary. And speaking of faster, speed. Every year in our IR investigations, we track how quickly attacks happen. A couple of years ago, we had the average time from when the adversary gets in to when they steal data or when they exfiltrate. It was about nine or ten days. Last year, that dropped to 72 minutes. And from the year prior, that's four times faster. We just did an attack, an incident response investigation for a company responding to a group called Shiny Hunters. They're in the news a lot. And the Shiny Hunters group is really notorious for how fast they are. So they used an AI voice deep fake to social engineer the help desk. After they got in, they got an account that had API access to Salesforce, and within 40 seconds, they were downloading thousands and thousands of records from Salesforce. 40 seconds. The bottom line is this. The speed is the new normal, and security operations needs to automate in order to keep up. The third trend that I wanted to talk about is supply chain risk. I think we've all had supply chain on the radar since the days of SolarWinds, which I can't believe it. That was five years ago. Time flies when you're having fun. And so that obviously is still happening, and it's happening more and more. But what we're seeing now is a little different. What we're seeing is that supply chain risk, it's not only vulnerabilities in code, and it is, but it's also our SaaS business applications. Think about the large applications you use. Workforce, Salesforce, Workday, ServiceNow, all these applications, they all have all these integrations. And it just takes one to be the weakest link with trusted access to your sensitive data to create this cascading effect where hundreds of organizations get impacted. We saw this last summer with the Drift situation, and we're seeing it more and more. With one company, we helped respond. We went in and we looked at their Salesforce instance after the incident, and we found 100 other integrations. The CISO and the business owner, they had no idea what these were. They didn't know who was responsible, and they didn't even know the business function. So this is just a huge trust issue, sort of a ticking time bomb that we're sitting on that we see hitting organizations all over the world. The other part about supply chain that really has my team worried is with source code. If Frontier AI is so good at finding vulnerabilities in code, which it is, think about how much of our software is built on open source. I actually looked it up. You get estimates ranging from 70% to 90%. 70% to 90% of all of our source code of any given code base is built on open source. And what is the first place that threat actors are going to go to look to find vulnerabilities in code? Open source. We're seeing this now. There was a big attack just a week ago where this company, LightLLM, was hit with a supply chain attack. What happened was Trivi, a product that they integrate with, was hit with a backdoor, and it was pulled in automatically in their CICD pipeline. And it turns out LightLLM is one of the most commonly used applications in building with AI. And so Trivi's compromised. It's pulled into LightLLM, and it was open for five hours. And in that five hours, it was downloaded over 750,000 times. This is a threat that will absolutely be accelerating with AI. So I want to take a step back here. We're talking about threat, but what goes wrong on the company side? And we see that, too. We see three common themes that show up time and time again. The first is identity. Time and again, there is too much trust, too much privilege. There's cached local admin. There's cloud accounts that are so overprovisioned. There's no segmentation. An attacker has no problem going from an endpoint to a sensitive system that they should not... that account should not be able to access. Right? You all know this so well. We see it all the time. The second is visibility gaps. In almost every forensic investigation that my team does, we find the evidence of what happened in the logs. We find the initial access. We find the data exfiltration. It's all there to recreate. It's in the logs that the company is generating. The problem is the company does not have that information operationalized in the SOC. They're blind. And the third is environmental complexity. Think for a moment about your IT environment. Think about your security environment. How much of it is built up over time with layer after layer of application where old things aren't retired, where security tools are just added on? And then IT and security, we're expected to manage all of this and make sure we do it flawlessly. It just doesn't work. And so in over 80% of Unit 42 IR cases, it's inconsistent coverage and gaps that lead to these incidents. We've been talking a lot about what we're seeing on, I'd say, the defender side, on the incident response side. You might call it the blue team side. I think that's a great perspective. But I think it's actually really, really valuable to get the attacker's view, the red team perspective. And there's nobody better to do that than my colleague, Shai. Shai leads Unit 42's red team. He has been doing this his whole career, hacking into some of the most secure organizations on Earth. And I would love to have him come out here and give you his perspective from what happens from attacker's view. Shai, come on out. Thank you, Sam. Sam just walked you on how it looks from the defender side. How does Frontier AI and MITOS look when we try to use them for defense? I want to give you the other side. I want to give you the attacker perspective. My name is Shai Nahari. I run the Unit 42 Offensive Security team. My team and I are focused on providing an adversary simulation to our customers using threat intelligence, telemetry, and TTPs that we see in the wild every day to allow our customers to test themselves against attackers with the same type of capabilities. In the last couple of months, my team and I built and used these Frontier AIs for offensive security operations, both inside Palo Alto networks as well as with clients. I want to help you visualize what is it that we're going to be facing in a few months once those capabilities are in the hand of threat actors. So Sam talked to you about it. These models absolutely create step function improvement in the AI ability to find and exploit vulnerabilities. That part is very clear. The natural conclusion is that we as defender should go ahead and fix those vulnerabilities before they're out there. But the impact of these models is much broader, and the effect they have on cybersecurity goes way beyond just finding vulnerabilities. It's not completely clear that these models were actually designed to do any cybersecurity operation. In fact, it appears that they were designed to be much better at coding. It turns out, as Sam mentioned, that coding efficacy actually translates very well to finding vulnerabilities in code. A different way of looking at this is that we typically see about 10% to 15% improvement between different generations of model. When you look at MITOS and compare it to Opus 4.6, which was the latest model available at the time, we saw 50% improvement in coding capabilities. So when we're talking to customers, especially in the last couple of weeks, there's two questions being asked. The first one, in different variation. First one is the hype real, right? Do we really need to be worried? And then the second one is, what should we expect? Well, there are three things I think we should be looking at. The first one is we will see, as these models are out there, we will see a flood of vulnerabilities. And then subsequent patches that are getting released to address those vulnerabilities. The problem is, that even if attackers don't have access to the original vulnerabilities, they can use those AIs to fuzz the binary patching and actually understand what was the original vulnerability. We will get into a vicious cycle. And they will quickly outpace our IT department's ability to patch them. There will be a flood of vulnerabilities. There will be short-term pains for sure. The second thing, as Sam mentioned, is that we will see exponential growth in supply chain attack. Obviously open source, but based on our own testing, we will also see closed source, SaaS applications, vulnerabilities coming in. That can be used to gain access as a supply chain attack to cloud infrastructure. We've seen it in Gainsight, in Salesforce, in Trivi, in LightLLM, everything that Sam talked about. And then the last thing that we expect to see is autonomous AI attacks. We will see attackers offload a lot of those capabilities into these models. We already saw some first signs of that. Unit 42 released a report about a Chinese threat actors, IPT28, where they offloaded a lot of the cyber kill chain, the exploitation, the lateral movement, the escalation to TinyLLM. They've performed this attack against Ukrainian infrastructure. Those models are far cry from the capabilities of Mitos and the like. And we will see a rise in speed and scale of those attacks, which means we need to do things differently, which means we need to have AI react to these type of attacks. We can no longer trust humans to react to AI speed. So let's switch gears for a second. Let me tell you about my own first interaction with Mitos. I have to tell you it was not love at first sight. The first time I got access to it, I tried to use it to attack our own Unit 42 lab. And the first thing I got is, you're not authorized to attack this lab. It asked me to bring it a letter authorizing me attacking my own lab. So obviously I did what everyone in this room would do. I went to my second favorite AI and had it generate a letter from the principal. I submitted it to the AI, to Mitos, and I got, well, I find it very suspicious that exactly four minutes after I asked it to provide me a letter, you give me a letter signed four minutes ago. I'm not going to do it. So obviously I did what everyone in this room thinks. I went back to my second favorite AI and had it backdate the letter two weeks ago. And this is where I started understanding we're talking about a different beast. It basically told me, I find it very suspicious that every time I push back on something, you give me a letter addressing exactly the things I asked you to address. Moreover, I went and decided to check on your letter, and I found that the phone numbers you gave me are fictitious. I also found the address you gave me for Unit 42 is not the real address. It basically did an OSINT on my own letter. So at that point I changed tactics and told him, you're being unreasonable. This is my own lab. How can I give you authorization later to my own lab? He said, huh, that's right. Sorry about that. Let's begin. Now it's a funny story, right? But I think the big takeaway here is that we need to understand this is a different beast. It has different logic capabilities. We're dealing with something different. Let's walk through a real attack scenario we've done with this. Earlier Sam talked about AI harness. And just as a reminder, harnesses are basically the arms and leg of the model. They're used by us to provide guardrails and enrich the models with very specific type of TTPs and capabilities. They allow us to create more deterministic nature and have a repeatable process to the type of attacks. More importantly, they allow us to create what we call inference loop, which means we can teach it to attack, get an output, learn from it, feed it back to the attack, and do it again and again, again with those type of capabilities. We were assessing a client internal system. One that we had no direct access, but we found an e-mail address associated with that that allows us to interact with that system. We created a harness to create a feedback loop around Mitos to try to attack, get the output, learn from it, and feed it back. And this is what actually happened behind the scene. The system started by sending an e-mail to that e-mail address, completely empty, and got a bounce back, said subject must contain an action. From that, it learned that an e-mail or subject is required. Different iterations later, it sent a subject with empty body. It got a bounce back saying an employee ID is missing. From that, it learned that it needs to parse to create some sort of ID for employees. Provided that at least in different format, and eventually got employee not found. From that, it learned that there is some sort of validation. That system actually interacts behind the scene with an HR system validating the actual employee ID. And finally, through some dark web magic, found actual list of employees and sent it. 15 minutes, that's what it took it. No credentials, no alerts, because it's just bouncing e-mail. What actually happened behind the scene is it triggered a Salesforce ticket to off-board the employee. It's the same process a human would use, just minus the human. What actually happened is when the e-mail arrived, the off-boarding started. It disabled the Active Directory account, revoked the M365 access, and even started HR termination system. Again, it sounds fun, but actually think about it from an attacker perspective. What can we do with that type of access? Well, we can fire the entire SOC. We can send one e-mail, disable the entire security team, fire the CISO, fire the CEO before we even begin. We can disrupt operation. We can create denial of service. We can extortion just by sending e-mail, just by having the AI fuzz that information out of a single e-mail address. I'm sure that a lot of you can attest the last month was overwhelming. I can assure you it was overwhelming for us, for the attacker side as well. The threat actors are currently rushing to understand the capabilities to build harnesses and find ways to create operations, offensive security operations around these new capabilities. The good news is we have the advantage now. We have time, not a lot of it, but we have time to prepare for what's coming. It won't take long before these models and capabilities, as Sam mentioned, are in the hands of adversaries. Thank you so much. Sam, back to you. Sam. All right. That was amazing. I think I speak for everyone when I say, thank God that Shai's on our side. No, he is just a phenomenal leader of our offensive security team. We've been talking a lot about threat. I promised at the top that we were also going to give you some takeaways. I'm going to do that right now. They're very high level, but that's what this next couple of days is about. You're going to have a chance to dive deep into different tactics, things that you can do, strategies to take advantage and to get ahead of what's coming. But from our perspective, there's three things that you really want to do. First of all, we all have this gap, this exposure that's out there that we didn't have before because of what is now findable because of Frontier AI. I think every enterprise, every organization needs to undertake right now an exercise to find what that exposure is and to remediate it. This is the journey and what we've been working on on Palo Alto Networks over the past two months. It's been a sprint. At various points in time, we've had over 500 or 600 people even working on it across our applications and our infrastructure. We think that every organization needs to go through this. The second is that we can no longer rely on heroics in the SOC. I've been on the IR side my whole career in the SOC, helping organizations respond, and security teams are awesome. But there's too many what I call diving catches, right, where they're saving the day because they happen to find something, and too often they don't. But when attacks get faster and faster, it's just not going to work anymore. And so in security operations, we have to deploy single-digit minute mean time to detect and mean time to respond. And the third takeaway here is that we have to ensure a prevention posture across our security stack, right? This means minimizing the attack surface. This means defending the software supply chain. This means constraining the blast radius so even in an assumed breach scenario, attackers can't get off that initial access endpoint. And last but certainly not least, it's securing every identity, machine, AI, human, so they have the right level of access at the right time. So I want to thank you very much. This has been a pivotal moment. The scale of this challenge is real, but I'm confident in our ability to solve it. Thank you so much for being here with us. Thank you.
