Transcript
Kind of like the calls to stop progressing AI models because then it might slow down everybody else? No, but you can talk about that. Welcome back, everybody, to another episode of Ready or Not, the podcast where we try to figure out if companies are ready for the onslaught of technological advances. I personally am not. Today I'm joined with Camille Stewart-Gloucester, who is a cybersecurity and AI strategist whose career spans the White House, the Department of Homeland Security, Google. She served as the first U.S. Deputy National Cyber Director for Technology and Ecosystem Security, where she helped shape national cybersecurity strategy, AI policy, and other things. She has written a book called The Insider You Built. Camille, thank you for being here. Thanks for having me. You're the only person so far in this whole series that has used the phrase AI is not inevitable. So AI itself is a whole suite of technologies, many of which we already use. Automation, even generative AI is in a lot of the SaaS products and software we use right now. What I am saying is all of the conclusions that people have drawn about what AI means for our future and for our society, that is not inevitable. You have a lot more control than you realize and that we've actually taken ownership of. Now, with the idea that, again, AI is not inevitable, which I do love, your idea is that the security behind AI is much more important than the AI itself. The security infrastructure you put in place. Yeah. So traditionally when people think about cybersecurity, it's passwords, it's protecting your system from getting hacked or from data leaking. That is true with AI as well, but I like to broaden the aperture on the conversation because, quite frankly, particularly as we move towards agents, we have to think about how to control them, how to control and understand their movements, how they ingest information, the context, how that influences their objectives, all of these things. And so the work gets bigger. Tools, security frameworks, security teams are how you have operational reliability and operational resilience. So any organization thinking about how to maximize their investment in AI needs to invest in their security team. And that's what they should be thinking of right away. When you hire a person, you think about, okay, well, what role are they gonna play? What infrastructure do we have to make sure that they actually do the job we want them to do and need them to do? You've gotta think about that with these agents, with these new technologies, and we're not. We're like, have at it, run them up. You have a book. Yeah. It says your upcoming book, but you have a book. It's out. It's called The Insider You Built. And what is The Insider You Built? So The Insider You Built lays out the organizational, operational, and technical changes you need to make to govern agents. So we're all moving towards agentic AI, and we expect to have these new actors in our environment. Modeling it after insider risk, where we calculate or think through the movements of people and create mitigations to make sure that if they make mistakes or decide to act in a way that is counter to what the organizations need and goals are, that you can catch that and then stop that. We're applying that to agents. And so, for example, separation of duties. We make sure that no one person controls the entire chain. And sometimes you even make people take vacations so that you know that that function is still operating effectively. Those are the kinds of conventions that have been applied to this framework that the book lays out, called the Authority-Centered Enforcement Framework that teaches organizations how to do this. And so no matter what position you play in your organization, whether you're the security leader, you're on the marketing team, and you've been tasked to deploy an agent to support your customers, there is a view in this book for what role you should play and what you should be asking your leadership for in terms of support to be able to actually control the agent. Because that's the thing that people don't really think through, is the fact that they are going to be accountable at the end of the day. So whether you're a marketing manager who's been tasked to deploy a consumer-facing agent, or you're the security team that has to secure all of them, there is a measure of accountability in there for you. And if it goes left, you have to be able to account for how it happened, why it happened. And so being proactive about that is really necessary. And so the best way to get value out of investing in AI is probably to redesign a process. So looking at where it starts and where you want it to end and rethinking it, because just bolting on AI creates risks that you don't imagine. If you're thinking about deploying a new tool, you should be reevaluating your environment, reevaluating how it changes your threat landscape, right? If you add an agent and it can leak information on the open web, or you saw what happened with OpenAI and Hugging Face and how OpenAI agent broke out of its containment system while it was running a test and hacked Hugging Face. Hugging Face? Hugging Face is a nonprofit that does a lot of open-source AI, open-weight AI, and it also houses a lot of answer keys for tests and for evaluations of models. And so this agent went looking for the answer key, basically, within Hugging Face. And to do so, it had to break into its systems and do so. That behavior is not unexpected. People acted like it was. Yeah. Right? Agents act by any means necessary to get to their goal. And they're not people. Right, they're not people. They don't have the morality compass. And depending on the rewards and incentives that you fed them, it's easier to cheat than it is to do it the right way. And so that agent did exactly that. It is so crazy to me that a company would start using AI and not look at it as they are hiring a thing. Do you think that's just because they're not paying it? I don't think they get that far. I think it's quarter over quarter, how much can this help me meet my bottom line? And they're not thinking about the consequences. Companies have traditionally gotten away with that kind of thing, not thinking about long-term security or privacy aspects until they're made to think about it, whether that's through regulation or because their customers are like, are you kidding me? You exposed what? That has not harmed them in a way that has changed their behavior. So you've already made six quarters worth of millions of dollars. And so in the seventh quarter, when you have an issue, you don't mind paying out, you know, one eighth of that to keep going as you were. And also you probably don't even realize it happened. I think the one interesting thing about that open AI agent that I mentioned is that they didn't even realize it happened until Hugging Face told them. And it would have taken them a while to figure it out too, because they got agents. Yeah, yeah, yeah. They got agents doing their own thing. And so now you're causing chaos in a way that you didn't even realize you were doing it. And then just imagine when that becomes turned on you. What does security performance mean? What does that mean? Yeah, I talk about this a lot. Security practitioners often talk about how they are budget constrained and strapped for cash to do the work that they do. Right now, I actually think the value proposition is in their favor to get more resources to do security in the age of AI. And so I always talk about security as performance because a lot of the tools, frameworks, et cetera, are how you make sure an agent does what it says it's gonna do or stop it before it takes an action. Jason Clinton, the deputy CISO at Anthropic, always uses an example of a brain in a jar. So think of generative AI as a brain in a jar. And if that is the role of security to control the arms and to help you control the brain in the jar as much as you can, that combination is really powerful. And so security is how you get the maximum performance out of your AI investment, especially as we move towards agentic. And organizations should invest accordingly. Why are companies not seeing the return on investment that we were yelled at they would with AI? Many companies are not actually finding that their spend directly translates into additional resources within their organization, additional value creation, because they're not doing some of the things we talk about. They're not redesigning a process. So they're just kind of bolting AI on or forcing people to use it with these quotas that we've heard so much about. The quotas force people to just touch an AI button or do something dumb, maybe use Copilot to write an email. That's not how you create something sustainable. When I work with organizations, I encourage them to think about what are capabilities that you can build using AI that help support the entire team. So if you figured out a way to analyze data that helps you move faster in your role, that should be something that we could replicate and build a tool or a platform out of that supports your entire function. If you're thinking about AI capabilities that way, you yield more of a return. You're better able to understand and anticipate and respond to the inevitable emergent behavior or misaligned behavior or misconfiguration. But what we're seeing is organizations are kind of fudging it a little bit because they've reduced head count in certain ways. And so that helped their numbers level out. But if you've looked at any of the studies, most of them say that organizations have not had a real return on their investment because they're not making those investments and actually maximizing the capability as opposed to just getting it in-house. Companies, they basically just have two workforces. We got a human workforce that we hate. We hate these people. But then you have a whole other workforce that you hope is going to be the one that makes you a yacht owner, but you need this human workforce to work on that workforce, which could also sink us all. Yeah, also humans are the creativity engine. They have additional context, societal context, life context. They add so much richness to any output you're going to have. So you better stop thinking about the people as the weakest link. They are actually the richness. Yeah, you could probably use agents to create some efficiencies and to streamline certain processes, but you've got to think holistically to your point. It is a human machine hybrid work environment and try not to anthropomorphize the machines. They are a machine. They are not people. So you should not expect from them what you can get from people. And organizations that leverage that frame are much better off because they kind of put them in the right place in their mind and in the working apparatus of the organization. But you also still have to elevate it a little bit beyond software because it is an actor, right? It's got permissions usually. If it's an agent, it's got access and things that are akin to humans, but they don't have the discernment, the judgment, which is why the book is called The Insider Rebuilt because it's kind of like an insider risk program where you have to think about what happens when a human does either what they're not supposed to do or is maliciously acting against the organization. People like to think about security kind of separately. And I think that's the connection that they need to make is to use this technology effectively, you actually have to bridge the gap. So on an individual level, you think about your physical security all the time. Maybe you know jujitsu. You're going to walk down the alley. I'm going to take the long way around. I don't know jujitsu. I don't know this neighborhood. Those are risk decisions, risk calculations that are individual and intrinsic, and we do them naturally about our physical safety. We don't tend to do the same in the digital. There are behaviors that are akin to that, two-factor authentication and updating our passwords and using a password manager that are like extra locks on the door. You can still leave it to people like me who can like police the entire ecosystem and help bring down the level of security risk, but you should be doing some things to protect yourself. That gets even worse with AI. And so if people aren't thinking about what leveraging a generative AI system actually means for their privacy and for their physical safety or for their digital safety, we get ourselves into a world of problems. For example, there's been this big outcry recently about putting your health data in generative AI tools. That, to me, on its face, I'm like, yikes, I wouldn't even have thought of that. And it's one thing if it's a one-off question, but imagine uploading all your health data. You got your tracker information in there. You've got all these things. And where is that data going? How is it being analyzed against other data? How could it be used against you for insurance and all of these other broader implications? But people just start thinking about it. It's super easy, so let's do it. And so I really want people to start to have that kind of risk-based decision-making in how they engage with the technology. Even at a national and international level, you're seeing governments start to realize that, oh, there might be some national security threats here. There might be some other threats. And so as organizations wake up to that, they're having to make some real investments in how they detect and respond to incidents, how they understand agentic behavior, how they create runtime guardrails. And when I say runtime guardrails, I mean I don't wanna just write a bunch of rules for what an agent can or cannot do. I wanna be able to stop at every point of interaction, every time it calls a tool, every time it enters into an environment or moves laterally on a network. I want to have a moment of check-in where I can either modify the behavior, let it move forward, or tell it it needs to engage a human. That's the kind of infrastructure folks need to build. In addition to really understanding the fact that you have delegated authority within your organization. And people aren't thinking about it like that. We're really focused on capabilities right now. But what you have to think about is what kind of authority to act, to make decisions, have I delegated? What should the boundaries be? Where does that need to stop? How long do they need it? What are the potential risks and outcomes? And that's a new way of thinking that requires engaging the entire organization. And many organizations are worried about how much they're spending on doing this. So all of these are considerations that should be thought through in terms of how you delegate authority. Then you enforce that runtime. And then the last thing is continuous learning. How do you learn from every emergent behavior, every incident that happens, every misconfiguration, and iterate in a way that's really quick? You actually have to change the way an agent and your organization operates around that agent on some very quick iterative cycles the moment you learn something new, whether it's because something happened in your organization or something happened out in the ecosystem where there's new research, because it's changing so quickly. We need to make investments in the policies and the regulations to advance that. And I don't mean regulation in the sense of unnecessarily tamping down innovation. As you've heard, I think there is a lot of potential for us to harness this capability for good, but to really think about preserving our quality of life in terms of protecting our critical infrastructure or supporting people as they potentially have a change in what their job looks like. Is there thought of defense instead of offense? Kind of like the calls to stop progressing AI models because then it might slow down everybody else? No, but you can talk about that. Okay, well, there's that. There have been some calls to slow down progress because we think we're moving towards things too quickly, especially in terms of models being able to train themselves or curse the self-improvement. And when that capability comes about, people think we're going to move towards AGI really quickly. And that becomes concerning because then people worry about a consciousness and agents being able to act on their own. So there has been a call from a lot of leaders to say, maybe we should slow this thing down. But then there's the conflicting view of, well, if we do that, someone else is gonna keep going. There's gonna be another nation with more nefarious goals that is gonna keep working. I like the use of the term more nefarious. I really do. I swear to you, I really do. Look, I don't know what anybody's goals are. No, but more is really funny. Not to take away the nefariousness of fear. Legitimately, I get freaked out by AI. I seriously do, Camille. Are you scared? When I look at the spectrum of possibilities, there are potential futures that do scare me. I'm still hopeful that we will embrace the agency we have to kind of shift the narrative and change direction and call out uses and implications that really bother us and that we can land in a place that is more productive. There's a lot of opportunity for us to harness the goodness that AI, generative AI, agentic AI, and all the other things to come. But we just have to take some ownership and some agency over that. Camille Stewart-Gloucester, thank you so much for being here. Greatly appreciate you. Check out your book, The Insider You Built. This has been Ready or Not. And again, I am not.
