Transcript
and cybersecurity. I'm your host, Charlie McCarthy, and each episode, we deconstruct the evolving AI security threat landscape from adversarial machine learning to the future of automated defense. Back in September, I had the privilege of sitting down with industry-leading expert Jim Revis, who's co-founder and CEO of the Cloud Security Alliance, for an interview while attending an AI safety workshop held here in Seattle, Washington. It was a very fruitful conversation, and I'm really excited to finally share it with this community. Let's dive into the Nexus. So before we dive into the contents of the episode, would you mind debriefing us a little bit on the workshop today, how you thought it went, and maybe some of the topics that were covered during it? Actually, before we get to that, maybe for audience members who are not as familiar, talk a little bit about Cloud Security Alliance at a higher level, and then we can dive into today. Sure. So Cloud Security Alliance, we're a global not-for-profit organization. We're coming on to our 17th year, and our focus is research, best practices, education, and then certification, both of individuals in their own professional cybersecurity careers, as well as cloud providers to make sure they're following the best practices. Now, as you can imagine, how things have changed from being called Cloud Security Alliance 17 years ago is about two-thirds of our work is AI, and we have this AI safety initiative, which is basically doing all of those things with AI. So that's who we are. We have about 160 chapters around the world. There's probably out of that about 12,000 volunteers, which is our superpower that are great researchers that really help create this. So we got one of the most powerful platforms to identify the problems, the challenges that are coming around the corner with technology, and to be able to come up with the solutions. I've been in cybersecurity for 30 years, and I tell people everything we've learned is just practice for what we got to do in the next two or three years dealing with generative AI. Yeah, that's particularly insightful and impressive. I don't actually think I had realized the breadth of CSA or how it had grown so quickly. I mean, that's an impressive number of volunteers and very supportive of kind of this all-hands-on-deck moment that we have, especially with the AI momentum that's happening. It's encouraging. Well, cybersecurity, it's always got that mentality of first responders and like helping out the community, and even some of them literally have come from those types of professions. And so it's really cool to see people like volunteer their time with this. Very cool. Okay. So, folks at CSA, your team has been tracking AI safety and security for years, presumably, especially since the ChatGPT release at the end of 2022. And it's hard to believe that so many years have passed since then. It almost feels like it was announced just a couple months ago. Things have been moving so quickly. Can you reflect for us on maybe some of the lessons learned over the past few years, but also even during events that you lead similar to the one from today, where like we're trying to coach the community about these pivots in thinking and also approaching, you know, how we're approaching risk now and maybe architectural decisions now that AI is so heavily involved? Yeah, absolutely. So when you see something that you know is going to be a sort of a seminal moment in technology and creating sort of a next generation of what we need to do, you get like, it's almost like a Rorschach test for people in our community that say, you know, some people it's like, I want to be in denial of that because I can see all the work that is going to happen. I maybe see, oh, there's great opportunities to transform how we do our business or how we do cybersecurity. And there's always that, okay, with anything that's new, what are the threats going to be that we're going to need to? So we sort of see the spectrum of all those different emotions and everything that has come out of it. What we've started to like understand is like to say like history doesn't repeat, but it often has an echo. And that this is sort of a supercharged version of what we originally saw in cloud computing, where you had individual entrepreneurs, innovative people saw, hey, I can take my credit card, I can go to a cloud provider, and I can just create some new things. And then you saw people who would say that, okay, well, I've got to understand that the world's very different now. And it's very complex. And I might be working with one company that's in the cloud. And it's got layers of different companies, technologies. If you're sitting on top of an AWS, there could be 5, 6, 7 companies that are delivering this application. So you've got to understand the sort of shared responsibility or even sort of shared fate if we think about it that way. So the world's completely moving to this new threat models, new companies that are going to be your partners, brand new ways of ideating how you do cybersecurity, how you do threat hunting, how you do everything, new ways we've got to think about how we protect the information, because it's going to be in these new formats. But basically, it's sort of that same model where, hey, some of it's public, some of it's we got to do privately, we've got to do the vendor assessments, and we got to understand that sort of shared responsibility. So it's just moving so much faster. And like one thing I could say, this was an observation I had with one of the most brilliant people I know that has mastered AI, that I've been talking to about different projects and doing some transformational things with AI. And it's like building a house, and the specifications for the electrical changed three times in the middle of the house building process. The nails are like five different iterations of it. And so you see this sort of spinning your wheels as well. And okay, do I wait for the introduction of the next model or the next thing? And you just you have to dive in because we got to solve problems now, we got to build solutions now. But you have to have this, I guess, one really interesting sort of way of thinking about things is you don't think about when you're building something, what is the technology look like right now? But what is it going to look like in six months? Because if something's not working, like right now, you can imagine certain problems, oh, that's just a matter of a little bit more scaling, a little bit more reinforcement learning, those sorts of things. Mm hmm. Yeah. Excellent point. And I want to take back on a phrase that you used just a moment ago that stood out to me today. I think you also mentioned it to the workshop attendees, that shared responsibility model, which I very much buy into, and I would love for our audience to hear a little bit more about that. But just calling out, when we talk about shared responsibility, the different roles or organizations that are involved in that. So I think I'm trying to remember the slide that you presented today. But there's, of course, the creators of these foundational models, like the open AIs or whomever that that bears some responsibility for safety and security. You've also got maybe even procurement teams at organizations as they're scoping their AI use cases, you know, they've got a little bit of responsibility as well. And then practitioners within the organization, like AI developers, data scientists, legal teams, in addition to the vendors themselves that are actually using some of these foundational models in their products, and then passing them on to the consumer, who also has a little bit of responsibility in making sure that AI uses remain safe for the public and that sort of thing. So like, did I leave any out? I'm trying to remember it from that list. But like, Yeah, that was that was really good. I mean, I would say like, I think about it in like, on one side, I look at it as like the technology provider stack. So you would say, like you said, you've got the model providers that that's really core, they're building these LLMs. Then you have the cloud providers, like the hyperscalers, where actually, if you've got like a private instance, it's the cloud provider that's like securing model weights and things like that. And so it's their responsibility on that side, because they have the company built the model, then you've got like the orchestration pieces like Lang chain and other things that like make it like easy to use different sorts of models and do the rapid development. Then you've got like, if you're a big company, maybe you've got 10,000 SaaS companies. Now they're all SaaS AI companies, and they're using it. And so you need to understand because it's not eminently transparent, like what they're doing beneath that. So and then you've got like, the consumers of this, ultimately, that they have to go see, hey, is this this solution that I have? What are all those layers underneath? Which layer am I like, procuring and then make sure to do that, right. And then like you said, the other dimension of it is you got to look at all the different stakeholders within your organization, like this might be for a business unit, but you got to have finance involved, you got to have your third party risk management on the procurement side. They've also got to be looking at this legal a lot of time needs to get involved. There's so many like, interesting things where you need to understand regulators are going to regulate different countries are going to have their own rules. And so the idea of like, if you're a big company, sovereignty, and being able to keep information in a certain area gets really complex. So it does get really complex, and probably will continue to grow more and more complex until it starts to level out. Because as we're talking about, you know, regulations in different countries. Something we've talked about with some of our legal guests or attorneys that have come on the show is like, yeah, you do have to take the considerations or laws that are put into effect in your own country. But so many of the technology providers these days serve a global market. And so if your products are being used across the globe, like you do have to abide by other countries rules, and maybe some things that people aren't considering. Yeah, the shared responsibility thing is one thing that I could go on and on talking about and would love to pick your brain, because we have so many people in the audience. But like, an example of that, one that I like to use, and maybe you've got another one you can share with us is like a highly regulated industry, like maybe healthcare that is using AI to do help with diagnostics. Their responsibility in that situation, you know, once they have purchased this AI solution to help with some of this diagnostics work, is to make sure that they're not misdiagnosing people or like what happens if they do? They could face reputational consequences, financial consequences, eventually regulatory consequences. But I don't, correct me if you have other thoughts here, but I haven't seen where there's been enough of a precedent set for healthcare or other highly regulated industries to deal with the consequences of a failed shared responsibility model, like, who do we blame? Does the healthcare provider at that point have ground to stand on to be like, well, it misdiagnosed because the model was the problem, or, you know, like, it's complicated. Yeah, we certainly have, like, you can just predict, we have several years ahead of us, a very interesting case law and litigation that's ultimately going to decide how a lot of this works. On the one hand, there's already plenty of regulations and laws that deal with malpractice, for example, and misdiagnoses and things like that. And the idea that I think we all believe in right now, we're at this state with AI that it's critically important to have human in the loop. And so the human ultimately needs to be documented to have said, I reviewed that, I understood how the AI system came up with that diagnosis. Like, all of the lab work, it went all the different, the methodology with how it did that and say, I agree with that. And, you know, we've had things in, like, in law that someone has submitted a legal briefing and it's been hallucinated because, you know, some intern, they wanted to go, they wanted to go out to dinner early, go to happy hour. And so they use Chad GP to do that. The thing is, it's, we've got new ways of dealing with things, but there is existing law that deals with a lot of those things because, you know, frankly, it's not the first time that a lawyer has lied. And so that's just a different way that that actually happened. So it's going to be interesting where we're going to understand, I think there's some case law and liability. Okay, we actually need a little more prescriptive guidance, maybe through HIPAA and other things they're going to create with some more specific AI sorts of things that we'll see. But it's just really critical, like for all organizations that are, you know, part of their AI adoption strategy, it's the AI governance is so critical and to understanding that these systems cannot operate autonomously without humans understanding that output and the process and how all that works. Yeah, super smart. And you hit the nail right on the head with the human in the loop thing. And I appreciate you pulling me back from the like, sometimes I get a little bit extreme when I'm talking about this kind of stuff and lean toward the fear factor, which is not the right thing to do. But to your point, there are already a lot of existing laws in place that will probably continue to build upon and use to set those precedents. Like I think someone in the workshop that you ran today was asking similar questions related to like, well, what about privacy? And what about the data that goes into an AI model? And how do we make sure, you know, there are privacy laws, there's GDPR. And then like on the back of that, California developed some that later us federally as the United States kind of built on. So we'll continue to use the things that already exist to help govern AI until we can get much more prescriptive about the do's and don'ts over the coming years. And we're learning and understanding like where we thought maybe there wasn't a privacy issue. Like if you're tokenizing data to go into a model, it's going to transform it. It seems like it's splitting up all that information, but they're finding unique ways. And, you know, our community, the hacker community, they find out, oh, you can actually do this sort of thing, this to make the model sort of infer an identity and it'll be correct. So we will find, I like to go back to Dan Geer said, I think in 1995, like the internet treats any attempt to protect data as a routing error. And I think we're going to have to come up with a corollary to that that says something about how like AI, how it is able to discover anything that has been transformed. So we'll find out about that. But it's a big, big challenge ahead of us. And it's, it's good AI against bad AI, probably to figure those things out. Yep. 100% agree. So speaking about human in the loop, if we kind of talk about more of the people side of this whole AI revolution, something else that you chatted about in the workshop today was CSA's launch of the Trusted AI Safety Expert or TAES certificate program. Do you mind sharing with this audience a little bit about that and how they might be able to get involved? Yeah, absolutely. So education is one of our core missions. And so what we really want to do is to enable cybersecurity professionals, risk management, GRC professionals to have the equipment and the understanding, the strategy, the framework and the knowledge about how AI works, how large language models work, how you can effectively create strategies, govern them, how you can understand the architectures around them, how you can have them deployed safely, how you can continuously monitor them, how you can understand the different unique threat environments like prompt injection, data poisoning, all those different areas. And so we provide this education and we partnered with Northeastern University, which I'm just sort of realizing that the CEO of Palo Alto Networks, he's a graduate of Northeastern University and I think he's a big supporter of that. So hopefully we can get him to go take the course, but it really allows us to have like this really long-term institutional expertise in education that comes from a university with our applied business knowledge and come up with a product I think that's really great. And one thing I was mentioning in our seminar is that it's really hard when something's changing so quickly. And so what we've done to enhance this education is we've come up with a prompt library so that for all of the educational modules, we have lots and lots of prompts and you can take a prompt that's related to some educational challenge or a project you might have. And whether it's today or tomorrow or two years from now, three years from now, that you will be able to get like state-of-the-art current information with the latest models on how to do that specific project or how to be educated in a more detailed way about how that works. So this is it. Yeah. And so CSA, we're offering this. You can go to cloudscarylines.org and you can find TASE and it's available online, self-paced education, online examination. And we'll be following up shortly with the instructor-led version, which is a lot more richer. We still find that that's like what people, they really value that as well. Fabulous. And we will include links to all of these resources that Jim is mentioning in the transcript, the show notes for this episode. So hop online for that. Cloud Security Alliance right now has a lot of really exciting things happening as part of its larger AI security initiative. As I was sharing with you just before we hopped on here, a couple of my colleagues participated in the development of the AI controls matrix as part of the volunteer group, which is really cool. We'll include a link to some of that work as well. One of the other sections of the initiative that I wanted to get your take on was the STAR for AI assurance framework. Am I saying that correctly? And wondering what that type of framework means for establishing trust between like AI service providers and their enterprise customers at a global scale. Is this something that those groups can use or who's it meant for? Yeah. So the STAR program, it uses actually our cloud controls matrix and now the AI controls matrix as the control framework. And the STAR program, which has actually been around for 14 years, Palo Alto Networks has got an entry in there. It's been assessed against this for cloud. And then pretty soon you will for AI as well. But basically it's structured in a way that you can do a self-assessment and you can also do a third party audit assessment with our third party auditor network, which are all of the ISO certification bodies and organizations also that do SOC 2s. And so this ends up being a really strong, robust repository of information that enterprises use as part of their due diligence. They'll go see, oh, are they in a CSA STAR registry as well as, oh, for the self-assessment, I can actually read their responses to every question and then sort of see how that lines up with my needs, my own risk appetite. Now, one of the new cool things we've added with it is something called validated. And this is actually using AI to score, act like an auditor of these assessments that companies submit. Nice. And we are just shocked at how well it actually works, how consistent it works. Even like I have a lot of auditor friends, I don't want to give them a hard time, but like you can have two auditors have inconsistent or differing views on something. You go to different models or you ask the same model this again, very consistent in the scoring that you get out of this. So like we're going to be entering this world with so much technology, so much laws that compliance is going to be a huge, huge burden if we don't do more of this automation. Some people are kind of like GRC engineering, things like that. So that's the whole STAR program. And the AI controls matrix ends up being the controls framework for that. And what we do with it is we sort of position like here's the operational controls you need to implement to protect the solution, the business that you're running. And then we map it to all of these higher level frameworks, like, you know, the EU AI act says, don't do high risk behavior. Well, okay. Can you give us like more specific examples of how we, what is excluded? Yeah. How can we do that? How can we secure things and change it from high risk to medium risk? And so that's where you need something like this, which I think ends up being really complimentary. We appreciate your team helping out, volunteer. We had hundreds that like helped build that and that's how these things happen. There's a great body of expertise out there. It was a huge undertaking. We actually had the opportunity to record a podcast episode last season with a woman named Marina, who I think facilitated some of the volunteer sessions and talk through some of the categories in the matrix. We'll also link that to this episode. But I remember during the conversation with her and my couple of colleagues, the immense sense of gratitude that I felt for Cloud Security Alliance. I mean, for all of the work that you've been doing over the years, but like this is a crucial moment and we need people paying attention to this kind of stuff. So knowing that you've got a huge group of volunteers, all of these experts that care so passionately about making sure that it's safe, secure, we're getting it right for the generations ahead is, it's pretty huge. I mean, y'all are humble about it, but it's a huge initiative. Yeah. It's funny, Marina, she works in our Athens office and there's a Greek proverb that we sort of like, that like a society truly becomes a great society when the old men plant the seeds for the trees whose shade they will never enjoy. And like, that's sort of when you're in a nonprofit like this, we think about it's a hundred year company or we're solving tomorrow's problems today. We understand that. And like, let's go do those actions. But really we are, we're sort of shepherds of the community and the community really does a lot of the really hard work. And we're very, we're very appreciative of that. And we're appreciative like the organization like Palo Alto who helps fund like some of these things that happen because at the end of the day, you've got to be able to do that in order to make this happen. Yeah. All hands on deck. I'm going to keep saying it. Okay. Let's see. We are running out of time here, but I think I probably got a couple more questions. Oh, as part of the AI security initiatives, I thought I noticed you shared today. So Cloud Security Alliance as part of this initiative has focused a lot on like white papers, kind of academic research, community fueled research. But I did see a note about diving into open source. So there's an upcoming project. Are you able to share anything about that now or? Yeah, we have a couple of projects in that space. We're finding that we need to augment what we do with white papers because this area is moving so quickly. So we've created this area called lab space. It's labs.cloudsecurityalliance.org. And it's to enable like open source projects as well as other just like off the wall ideas to have a place. And so one of the areas that's been spun up out of that is the model context protocol security resource center, because MCP is going to be this. It's going to be like the the TCP IP. It's going to be the the stack that we're going to have agents communicate with agents super critical, not a lot of security in it. And so we've we've got already spun up an open source model scanner and hardening tool, which we'd love to have the community go work with that. And then we've got another one called Maestro, which is a an agentic threat analysis tool with some code. So that could be integrated. I think I've heard of the Maestro. Did Ken Wong? Ken Wong. Hi, Ken. He's helped out a lot with that. So he built it. It was his idea. But he wanted it to have this sort of governance framework and have the CSA community sort of give it like make it real sustainable. But there's a lot of other things we want to do there. So our whole 2026 thinking on the the the road map for our AI work is securing the agentic control plane. And that means a lot of different things, but like MCP and agentic payments, understanding that whole area, what we need to be doing about that, all of the hardening. One of our speakers was talking about we we've got to go from least privilege to least autonomy in how some of these architectures work. So so we think a lot of tools, a lot of open source stuff is really going to be very helpful in that regard. Awesome. Okay, I'm going to wrap us up here with Jim, you do a fantastic job of lifting the community up and staying very positive and solutions focused and not kind of from my perspective, leaning into a lot of the fear, uncertainty and doubt that some folks like to sling around in the industry. That said, is there anything on the horizon that tends to keep you up at night a little bit? Maybe maybe it's the agent stuff or like you're at this really interesting intersection of like academia and industry and maybe a bit of government like national security. Any special insights there you can share? I see the ability to find vulnerabilities at scale right now as something I don't think we're prepared for. And I do worry about like the next six to 12 months, what the proliferation of new AI discovered vulnerabilities that are automated and how that is going to manifest itself in a lot of new O-Day sorts of exploits out there. So definitely want to sort of stay ahead of that as best we can, like communicate that to everyone that we do be on guard for that. And we need to have our defenses like really they have to be as good as they can on the heuristics and the behavioral analysis. And our industry kind of was built on, oh, a problem happened. Let's create the solution to that problem that happened, not the problem that's going to happen. And I know we've gotten better at that, but that's probably fair to pick one thing just on a lot of like sort of the research and people a lot smarter than me that are kind of looking in that is like vulnerability explosion will not be a lot of fun for any of us. All right. That's fair. All right. We're going to end it on a positive note, though. Before we wrap up for audience members who might want to deepen their knowledge in this space, key takeaway for them or somewhere you'd send them a CSA event. I mean, Taze, obviously, we want to tell everybody to go take that certification. But just say you're a CISO listening to this and AI is not new on your radar, but you're still kind of wrapping your brain around like, man, how can I figure out even what the problem is and understand the problem and then get my team on board? Like, what's your advice for this group of people? What's something they could go do today or this week? That would be a positive step toward AI safety. Yeah. So education, I feel like in a large degree is like needs to be the mission. So much of what we do and, you know, having this great credential, we're very proud of that. But I think that one thing we found that's like really interesting is taking our research, like our AI controls matrix or top threats or some of the most important research and having if they're a CISO, if they are an engineer, they're in risk assessment, taking some of that research, putting it in a prompt and then asking for something like, I need a maturity model to help me with this project. We stopped like writing like policy templates at CSA because we could say, take the core research, put it in a prompt, tell the prompt what you want. You got to be kind of good at those sorts of things and have it build you either a custom education program or a custom project roadmap or a policy that's for your organization. Go use the technology, but don't reinvent the wheel with it. Seed it with like our research. Go to go to our research repository. Take the topics you're interested in, put it in there and get it contextualized for your needs, your projects, your compliance challenges. Very good. Super smart. I'm going to go do that myself. OK, well, again, it was an absolute delight. I can't thank you enough for taking the time to talk with us today. Thank you to our show sponsor, Palo Alto Networks. And Jim, I hope we get to talk again soon. Thank you.
