Transcript
we're not mitigating, there's no C2s don't spontaneously develop. Those are created and managed by people, people with jobs, whether it's a nation state or it's, you know, a criminal enterprise. You know, you are mitigating a threat that is conducted through technology, but by people. And so it's always the people that end up, I think, you know, in many instances, giving us like the kind of the largest target. Hello and welcome to Data Security Decoded, where we deliver actionable insights to reduce data security risks and improve cyber resilience outcomes. I'm your host, Caleb Tolan. And in this episode, I sat down with Adam Karcher, Supervisory Special Agent in the Cyber Division in the FBI. We spoke about projects the FBI cybersecurity team is looking to streamline with AI, the importance of understanding an attacker's mindset, and how community-oriented groups like ISACs are game changers for defenders. Let's get into it. Adam, welcome to the Data Security Decoded podcast. So excited to have you on. I want to kind of start the conversation going back a couple years in your career. So earlier in your career, you led the Eurasia Cyber Operations Unit. And so I'd love to understand when you were working within this unit, what were some technical trends in terms of AI and security that you observed across Europe and Asia that some of the Western defenders still haven't quite translated into their day-to-day work? Okay, so for one, kind of a reframing. That was 10 years ago. So I think OpenAI had just formed as a company. So there really wasn't a ton. AI wasn't AI as we kind of know it now. LLMs really weren't a thing. They weren't especially a part of the pun of the public consciousness. But I can kind of frame that towards like some of the kind of operational things that we observed over the years. And it actually kind of pairs with a longer trend. I give talks occasionally to various groups. And one of the things that I like to focus on is kind of the idea of convergent evolution across all threats, both nation-state and like criminal. So like one thing I think really that kind of Western defenders have to understand about like the Eurasia threat, as an example, would be that these are operators that are working in kind of a campaign mode. These are, there's an entire ecosystem around these actors. And especially with the Eurasia threat, it's a very blended threat between kind of pure cyber criminal, kind of state-allowed and fully blown state-sponsored. And so I think the one thing that like Western defenders have to kind of like understand about that threat is that they're going up against an entire ecosystem. A very kind of modernized, a very compartmentalized, but in like kind of a kind of an anti-compartmentalized way. There are entire parts of that ecosystem that are dedicated to kind of access brokers. You know, and then they, you know, also ones that focus on kind of the initial intrusion, exfiltration activity. And so there's an entire long-term cadence that these operators act in. Defenders tend, especially, you know, Western defenders and kind of defenders in general, it's kind of very response-based. So there's an incident, open a JIRA ticket, address the ticket, close the ticket, move on to the next thing. I think, you know, when you're going up against an adversary that has like a kind of a dwell time in years versus a in and out opportunistic threat, I think you have kind of have to focus your defensive kind of operation cadence to match that. And so, you know, understanding that the threat is running a full-blown operational program against you, your defensive posture should be kind of operating as a defensive operational plan against the threat. So more longer term, you know, deeper, deeper thought towards like, you know, why am I being targeted? Like, what are some of the vectors in? What are the least, what are the paths of least resistance? Operating in that mode kind of switches you from cleanup to, you know, getting in front of the problem before it becomes a problem. Right. And it's a big challenge too, because oftentimes these are like organized groups that are operating on a different set of rules. They aren't operating by the set of standards that you have to, or the same processes that you have to, they can kind of break and make their own rules as they move along. So really, really great point there. And kind of moving into more of your current role, you are a part of the FBI's AI working group. And I know there's a lot of mystery behind government using AI and how that, how kind of some of the techniques and opportunities that you have aren't always, they're a little bit on the forefront of what private industry is going to get to do in probably the next year or two. So in the testing that you and the team are doing, what are some of the use cases that really excite you about AI and how your team is adopting it that when it moves into private industry, even more so, what can we learn from? So we're more, so we're always looking at ways to deploy technology. That is a kind of an FBI, kind of a core tenant. I mean, we were, the FBI was largely created in response to and definitely shaped by technology, whether it's the invention of the automobile or the telegraph. So we are trying to be as thoughtful and respectful of our utilization for it. As for specific use cases, we're always looking to find those use cases that have like the potential greatest impact with the least risk, or it's more precisely the justifiable or quantifiable risk. You know, so moving from an AI that kind of just answers questions. So with the chat GPT or something of that nature, you ask it a question, it gives you a response. That's one thing. Moving towards agentic, right, you know, this is where everybody's moving now, that is a completely different scenario. You're moving from something that can take an action independent of a human in the loop. And so, especially when, you know, you've got them, you know, a chatbot that hallucinates a gremlin is one thing, and that's kind of big in the recent press, but a chatbot or an agentic AI that hallucinates a problem and takes an action on that problem, whether it's there or not, that's where you have to be very, very careful. And so a lot of the things that we are looking at are kind of use cases where they're well-bounded and they're auditable. So you can see where the decision, you know, might have inflected, you know, had some inflection point to a place where you need to like rein it back in. So like triage, whether it's triaging case files, or if you're a defender, like your, you know, your SOC alerts, things where volume, where it's relatively pedantic, but where volume is overtaking, you know, an investigator's or team's entire capability to like process and understand it. Those are the things where I think, as we do baby steps up along this gradient, those are the ones where I think we're putting a lot of our initial efforts. So vulnerability discovery is another one that's also big in the press. But a lot of the things, it's very, it's bounded scoped problems that an agent or even just an AI can handle and then hand off to a human. The things that, you know, obviously we don't just wanna flip the switch on are an agent that is given enough autonomy to react to a problem without someone verifying that problem actually exists. And so those are the things where, I mean, these models are so, they're opaque. And for a lot of, you know, whether you're technical or not, there tend to be opaque, like the topology of vectorized meaning is something which is, even for the frontier labs, you know, they have to tweak these things and they don't always get the response they're expecting. A lot of this is unexpected. So accounting for that kind of variability and output is something that as you're building your program and as we're building our program, we have to take into account. So we do this in like a gradual and informed and justifiable way. Right, absolutely, absolutely. And you mentioned vulnerability detection. And of course we couldn't get away from speaking about Mythos if we wanted to, but obviously there's a lot of concern within industry about Mythos and its capabilities in terms of vulnerability detection. So looking at old complex databases and finding exploits in legacy systems is a major concern. So if Mythos can read and dissect that architecture better than the original developers, what is the best path forward for defenders than a total system rehaul? So, you know, some of the legacy systems have been a problem for a while. And sometimes it's really hard to define the risk when things just work. Like when your banking system works, it works. You're not gonna complain about it. You know, when some mainframe is doing the job it was designed to do, I know it's hard, especially in, you know, not everyone has an infinite budget or an infinite resources. And so, you know, where do you put that switch to like we have to rebuild? The system is way too critical risk to keep up and running. Now, I think the interesting with Mythos, and I haven't had any necessarily personal experience with it, is just like any tool, like all tools are dual use, whether it's technology or not. Like a hammer can be destructive or constructive, depending on the wielder and the wielder's intent. And so, you know, like search indexes, Google and a bunch of kind of enabled OSINT, social media enabled influence operations and target tracking. And so, you know, an AI is the same way. It's just accelerated. And so I think some of the things that, you know, an organization can do to kind of manage that risk is to understand that just kind of like the way the Eurasia cyber threats are running kind of sophisticated campaigns against you and AI just accelerating and enabling that. Looking at defense as just kind of offense running backwards is kind of one way to think about it. Like any tool that you would use to better defend or better understand your environment, the adversary is using to better target and better penetrate your environment. And so that's just kind of the nature of technology. Being able to, like some of the best cyber agents I've worked with are the ones that can wear both a defensive and offensive hat simultaneously, or at least switch between them quickly. It's being able to think like the threat actor in order to get in front of the problem versus kind of batting cleanup or doing the post-mortem, which, you know, we're all, unfortunately, a lot of our cases are post-blast, as it were. And so doing the things, like treating your defensive operations like kind of the inverted offensive campaign, I think is one way to get out in front of that problem. So it's running kind of adversary emulation as part of a matter of fact. You know, threat hunting from the threats perspective, like where is the path of least resistance? And now with so many interconnected capabilities, SaaS and cloud-based everything, it's no longer just the path of least resistance, it's multiple paths of equally low resistance. So, you know, getting a handle on that, I think whereas AI can kind of plug in to the volume and volumes of data that you're gonna have to like collect and integrate is gonna be key for defenders. So like, you know, training like this is, you know, kind of a permanent and ever-evolving risk is key. And so whether it's for legacy code, I mean, it's gonna be, it's always hard to articulate that risk. You know, when things are working, like when you're counting beans, the beans that sprout get all the attention. So, you know, demonstrating to your management or up your chain to your CISO that like, while this is working, you know, it is a critical, critical risk. That's just like always a hard thing to do. So kind of getting, treating your, those risks, like as the threat is trying, as if the threat is trying to exploit them is I think one way to kind of get in front of that problem. Just kind of do that kind of risk-based planning, have a clear path forward and be able to articulate that to your, you know, up your chain in order to get these changes done. Right, right. And I totally see your point you made too about training and like tabletop exercises that like how threat actors are evolving, their techniques are so dynamic that constantly stretching that muscle and exercising both your kind of red team hat and your blue team hat at all times is really going to be the best way to see progress on your resilience. And, you know, we had a really interesting conversation on the podcast with Cynthia Kaiser just a couple of weeks ago where we were talking about threat actor behavior and how that's been evolving. And so that's kind of something I want to kind of have you elaborate on too. So, you know, everybody's talking about like the technical side of groups like Scattered Spider and, you know, all of these other threat actor groups. But when you're tracking a group that isn't necessarily a nation state, like I'll use Scattered Spider again, they're not a state-backed actor necessarily, what are some of the digital breadcrumbs that usually leads to some type of action, whether it's like in a law enforcement case, an arrest or, you know, some type of action? Is it the code that they write? Is it something a little bit more anecdotal? What are the things that you're tracking to kind of suss out who is the attacker behind this action and this incident? So it's almost never the code. While it's kind of the most glamorous part, you know, in movies, there's, you know, there's always, it's always some amazing bit of, you know, bits and bytes that do something nearly impossible. But it's usually the people, in fact, it's almost always the people. You know, I'm giving kind of talks over the years, and, you know, it's, we talk a lot about, you know, command and control and like the vectors in and things of that nature. But you're, you know, when we mitigate a threat, you know, mitigate with a capital M, we're not mitigating, there's no C2s don't spontaneously develop. Those are created and managed by people, people with jobs, whether it's a nation state or it's, you know, a criminal enterprise. You know, you are mitigating a threat that is conducted through technology, but by people. And so it's always the people that end up, I think, you know, in many instances, giving us like the kind of the largest target. Yes, you can, you know, you'd be able, it might be able to trace, you know, an intrusion chain of events through reconnaissance and exfil and that nature. But it's usually the kind of the OPSEC mistakes of, you know, the people behind the keyboard that give you your greatest wins. Like in order to be successful, you have to be, you know, have some technical discipline, operational discipline and personal discipline. And usually people are good at like one or two, rarely are they good at all three. And the ones that have kind of that outsized blast radius are usually the personal ones. You know, whether it's, you know, you're doing scrolling through social media and you click on a post of someone we're already tracking, you know, that particular actor may have exposed themselves. You reusing a forum handle, you know, that allows it to trace back to either previous activity, infrastructure that like, again, these people, you know, no one has infinite time and infinite resources, burning down and spinning back up infrastructure isn't trivial. Infrastructure reuse, even if it's many, many years later, can highlight those opportunities for us in law enforcement to identify that activity. Especially, you know, previously before, you know, before domain privacy was a thing, registering at, you know, some old domain that you, you know, you used 10 years ago with your personal email address is the kind of that OPSEC mistake. And so humans are always squishy and squishy is good in law enforcement. It gives us a way to identify and pursue like those threat actors. Right, right, absolutely. And I wanna ask you a little bit about private sector, the private sector and partnering with these organizations. And so, you know, there's this ongoing conversation that many of us have had for years and years and years about public and private sector partnerships and information sharing. And you've been in the government as a defender for many years now on one side of this. And on the other side, oftentimes we hear like, oh gosh, there's like, I don't even know what resources are made available to me from the public sector. So how do we make these partnerships actually valuable for defenders and resource limited organizations that are honestly most of the times in the private sector or at like the state and local government level? So I think a lot of, you know, how we've pitched private sector engagement and is an incredibly important part of how we do our jobs many cases are, you know, land on our laps from a tip, from a partner, whether we have a relationship with them or not, it's usually a phone, it can be often be a phone call in that kicks off an entire investigation. You know, the standard kind of answer is like, you build those relationships before you need it. And that's very, very true. You don't wanna be like cold calling the FBI at two o'clock in the morning on a Friday because something happened in your network. You wanna establish those relationships and establish them early, you know, kind of maintain a good cadence of communication back and forth. You know, but those types that amount of slack, that amount of bandwidth doesn't always exist in at every level. So it might work for a Fortune 100 that has a CISO and a budget and, you know, conducts outreach regularly, but kind of the mid tier and lower tier in organizations that are, you know, that, you know, whether it's a regional hospital or kind of a municipal utility, they don't have those same budgets. They don't have that same kind of bandwidth in order to be able to do that. And so I think some of the ways you can, we can address kind of collaboratively how to make that make sense for everyone is kind of, you know, there's a few parts and on both sides. One is, you know, the FBI is not necessarily your first point of contact in order to maintain that regular cadence of engagement. Peer engagement is key. So, you know, working with your ISACs or your analysis organizations that like can keep you kind of in the loop with your peers and kind of working kind of the threat and kind of the landscape that way is one way where you can as a smaller shop with a handful of defenders stay engaged. Trying to engage around something specific, you know, just going for, you know, the coffee and donuts in the quarterly meeting or the, you know, the yearly meeting. That's great for coffee and donuts, but like find something that you need that you can engage with the FBI in order to kind of get that, you know, kind of get some of the meat back out of, you know, this relationship. So whether it's a sector brief or, you know, a conversation on a specific concern or indicator that you're seeing, just, you know, showing up every quarter or, you know, biannually isn't gonna necessarily kind of make those relationships work. And I think one thing which is challenging for us is kind of expect reciprocity. You know, it's always FBI, sharing the US government, not just the FBI, is often seen as kind of a one-way street. Thank you for your indicators. They were very helpful. We'll get back to you if we need you. And that's kind of the nature of obviously the cases we work. Not everything is just shareable instantaneously, but understanding how you can engage with the FBI, it might not be, we might not know you need something that might be easily shareable unless you tell us. And so not everything is contained within a vault, inside a vault. A lot of this can be, you know, us understanding what you as a defender in the private sector might need could help us identify ways we can get you that and satisfy that need. Right, absolutely, absolutely. I love the call out you made about the ISACs too. They're great organizations to find community, address those very specific issues that you are facing kind of in your niche environment. To your point, specificity is so key to make sure that you are making an impact on your organization and able to move things forward and not just showing up for the coffee and donuts. Awesome. And so, you know, I kind of want to return to another thing you mentioned earlier about AI-driven threats or really AI as a tool, I should say, and how it can work for the defenders, it can work for the adversaries as well. So looking more from the defender point of view, what are three specific actions that a defender can take right now to ready themselves for AI-driven threats from these kind of emerging new models and tools? The three things that I would kind of recommend is one, get your AI stack under control, or not your AI stack, I'm sorry, your identity stack under control. You know, identity is the way I think a lot of AI-enabled, you know, threats are gonna be able to get into your network, whether it's, you know, deep fakes or incredibly well-crafted phishing emails. And so getting, understanding that particular vector and getting a handle on it, you know, through various, you know, all the things you would expect, like phish-resistant, multi-factor authentication, constantly, like, auditing, kind of like your authentication telemetry, those things, you know, getting a handle on those, which, again, is something we've been saying for as long as I've been an FBI agent. Like, make sure you understand how an authorized user gets in because then you'll be able to detect when a non-authorized user, or someone that is, like, utilizing your, you know, has phished your account and is now, like, using your valid credentials to act on your network. That also includes non-human identities, you know, with APIs that are vulnerable. I mean, you've seen some of the stuff in the recent press about some new tools get deployed and people are leaving their API keys out for anybody with Shodan to search for those particular vulnerable deployments. That's gonna be, especially as we move into more agentic utilization, that's gonna be a critical piece, even for defenders. Like, having both your human and non-human identities under control is probably the biggest thing that I would recommend, like, moving forward. Another thing I've been saying since I've been a cyber agent is log as much as you possibly can. You can't see what you don't look for. So, you know, it is critical to make sure that, like, within reason, again, as I mentioned earlier, kind of, it's gone from path-based resistance to multiple paths of low resistance, like, which, of course, amplifies the amount of things you gotta track, but not investing in adequate logging is just gonna leave you blind. You know, absence of evidence is not evidence of absence when it comes to cyber threats. So not looking just means you're blind, doesn't mean you're not finding something. And I think the last piece, which I think is critical, which is also kind of calls back to kind of how our philosophy in the Bureau for kind of identifying opportunities for AI is make sure you understand where your human-in-the-loop sits now before it's too late. Whether you're deploying something locally as part of an agentic defense capability, you wanna make sure you understand where that kind of, like, authorization sits with your human-in-the-loop before, again, much like not making that call at 2 a.m. to the FBI in an emergency, you wanna understand where you are and how that kind of control scheme is deployed well before you need it. So those, I think, are kind of the biggest things that I could see for a defender moving forward. And they're all complementary to one another, identities into non-human identities and into visibility of your agents and what they're doing. And it's all complementarily, like you're saying right there, Joe. Just like the threat, it's a gradient. Everything's a gradient. Right, absolutely, absolutely. Now, I wanna ask you about two inconvenient truths. What are two inconvenient truths that we as an industry need to face to make sure that our private and public sector partnerships are the most valuable? That kind of calls back to one of the earlier questions that, like, it is, our information-sharing posture currently isn't symmetrical. It is asymmetrical. You know, and I think most people recognize that. You know, when we, when I, you know, doing outreach, you know, it's always, it's the same, I hear a lot of the same concerns that, like, you know, we've shared X, Y, and Z, but we haven't got anything back. And I think that gap is real. And a lot of it's on, again, as I, you know, stated previously, it's, communication's a two-way street. So while it, knowing what is needed helps us understand how we can, like, turn our kind of view inward to see how we can provide it. Whether, and it might not always be, you know, in the format or the way that a defender or a private sector partner is expecting. You know, we've built these partnerships, these private sector partnerships, you know, for the Fortune 100 companies. And, you know, we have to work, you know, most of the risk exists outside of that. And so, you know, it is going to be a concerted effort across, you know, both sides of this equation to, like, you know, support where, support those organizations that are actually bearing the brunt of the risk and how to get them what they need. And it's, again, going back to ISACs and other organizations that can help kind of be that, you know, the mid-level manager, but in a good way, for cyber threats. It's being able to get that, you know, that information where it's needed, when it's needed, well before it's actually needed. So these organizations can be postured as best as possible. Right, absolutely. So outside of the ISACs, have you seen any other specific community models be really helpful in your work or in kind of your network's work as well, in terms of information sharing, getting the right people in the right room, talking about the same issue so that some of these issues could be resolved? I think ISACs, as a model, work very well. And I know they, you know, they're generally focused on, you know, critical infrastructure. But I think, you know, having the same types of models applied outside of that particular, I'm not saying niche, but that particular kind of like way of doing business, I think can do wonders for, you know, for any organization. A lot of this, since cyber's not really geographically based, it can be hard to align, you know, sectors or victims in the same space. But that being said, you know, in your local communities, you know, whether, you know, there are people facing the same problems as any other particular threat. So I think, you know, engaging with your local field office to kind of like build those relationships. And, you know, a lot of it is proactive. So in Philly, we've had some opportunities where, you know, like-minded cyber defenders or technical teams would kind of like kind of self-organize around a problem and kind of stay engaged even outside of the traditional, like, InfraGard meetings. So I think a lot of that's going to be community-based. But those are the types of, it all comes down to relationships, knowing who to call and when to call them, but also having someone you can rely on as a peer. I think, you know, I think that's one of the better models kind of going forward. Right. That's so intrinsic to the cyber defense DNA, just like it's always been a very community-oriented community. And it's all about, like, making sure you have the right connections to help you when you need it the most and vice versa, too. You can be there for other people when, you know, they're experiencing something that you've been through yourself as well. Yeah. Well, Adam, thank you so much for your time today. What is the single most important message you want to leave with our listeners today? I think it kind of calls back to, actually, our first question or kind of our earlier conversations. Stop thinking of defense as kind of a reactive. Think about it as kind of an inverted off, you know, offensive campaign against you as a target. Those organizations that can build models around that, a lot of it, the talents there, it just has to be aimed in a slightly different direction. I think those are kind of the defenders that are going to be better positioned in the future, especially with AI accelerating everything so rapidly. I can't even predict what that will look like in five years. But knowing, you know, we've got decades of history understanding how a sophisticated threat plans and executes. So as a defender, understanding that all of these threats, nation state, criminal, and everything in between, are conducting campaigns against you, thinking about it as kind of a defensive, in a kind of a longer-term defensive strategy will position you probably better than the ones that don't. Right, right. And I have to say, if you were able to predict where we'd be in five years in terms of AI and how it's shifted the work that we've done, I think your crystal ball would be in pretty high demand. I think everybody would want to know where we're going to be in five years. So I definitely love that sentiment. You know, everybody's going to have their red team hat on at some point and understand how it impacts their work, even if they're not in a red team and a penetration tester in their day-to-day. So, Adam, thank you again so much for your time today. And until next time. Great, thanks for having me. That's a wrap on today's episode of Data Security Decoded. If you like what you heard today, please subscribe wherever you listen and leave us a review on either Apple Podcasts or Spotify. Your feedback really helps me understand what you want to hear more about and is one of the best ways to support the show. And if you want to reach out to me directly about the show, shoot us an email at data-security-decoded at n2k.com. Thank you to Rubrik for sponsoring this podcast. The team at N2K includes producer Liz Stokes and executive producer Jennifer Iben, content strategy by Mayim Plaut, sound design by Elliot Peltzman, audio mixing by Elliot Peltzman and Trey Hester, video production support by Bridget Kirkywild and Zorrel Joppy. Until next time, stay resilient.
