Transcript
Real stories, real defenses, and real recoveries, straight from the practitioners building and defending modern data environments. Hi, I'm Dr. Joy Purser, Global Field CISO at Cohesity, and welcome for another episode of our podcast, Zero Downtime. Today I have a special guest with me, Kristin Beneducci. Kristin, please introduce yourself. Thank you so much, Joy. It's a pleasure to be here. Great. Thank you so much for having me. I'm Kristin Beneducci. I was the head of security recently at a fintech, and prior to that, the deputy CISO at a public company next door. And as I think we'll get into a bit, I spent quite a few wonderful years at Sandia National Labs with the Department of Energy and a number of lettered agencies. Great. It's so good to have you with us, and your experience is really something that I think we all can learn from, so I'm grateful to have you on the podcast. Yeah. Great to be here. So there are two big topics that I would like for us to cover today, Kristin. I'd like us to talk some about incident response, because I know this is part of your operational background. And I'd also like to talk about artificial intelligence, or AI, and specifically about security guardrails for AI. So without further ado, let's jump into the interview. Great. What lessons from Sandia Labs and the private sector have helped you the most when it comes to incident response? So I think it surprises a number of folks, and a question that I get a lot is, how was the transition from Sandia Labs into the private sector? So this is such a great question to talk about those lessons from Sandia Labs, as well as how that translates into the realities of businesses. Hopefully not surprising, if you know what Sandia Labs does in the Department of Energy. The mission at the labs is security. It's national security. And of course, that's the mission of the business of Nextdoor and some of the startups that I've worked with and advised. It's to grow, and it's really the business, and we're thinking a lot about trust. So what do I take from so many years of working in a place where security is the mission, where everything is built around, including our incident response process, very well-oiled operation, everything is built around security, even when we're not talking about security, to a business where security is part of that? I think the thing that really stands out to me about the labs that translates is that rigor and that diligence, it's not the enemy of speed. It's actually how we get fast. The only, really the big difference, if I were to summarize going from the labs to private sector is how much we engineer, need to engineer that rigor, including having a very well-defined, well-practiced incident response process, other security processes as well, into the kind of fabric of the business. So much of the transition was about how do I take all of that rigorous security and do it faster and help the organization do it like a well-known muscle? You know, some of that is probably technology evolution, but probably some of that was culture change as well. So how did you address that in private sector? Yep, absolutely right. I think that's the, it wasn't a surprise, but some of the very tactical, it absolutely changes strategy when we're thinking about security as the North Star metric to how does security contribute to the business's North Star metrics, often revenue, customer engagement, customer trust. A few very tactical things that, you know, what that meant was translating security priority into the language of everyone else at, in the business. So for engineering, everybody sort of understood security was the mission. You know, building the culture at a company is, you know, helping engineering to say, to understand what high, medium, low, even, you know, what that means and how that relates to a high, medium, low, you know, pure engineering concern. So, you know, looking at the frameworks that other, that are driving prioritization in other parts of the business and integrating security risks, security levels, you know, some of the security, translating security terminology into how they think about prioritizing all of the other work that happens. The second thing I'll say is really anticipating time to gain alignment around security priorities. You know, when there's a prerequisite of security is kind of the known, there's kind of a given where, you know, we're going to, this is a party, we're going to do that. You know, I've used frameworks to, you know, to when launching new projects, just good project planning, and I budget more time to understand the different pain points of the business that are not security related and to create communication frameworks that will help others be aligned with the security why. That's the sort of cultural that happens when going from going to a business. You know, you spend a lot of time talking about the why and building ways to scale, to scale others, starting to be comfortable with security risk and having conversations about it. This is something that probably resonates with a lot of chief information security officers who are major advocates and stakeholders for a security strategy, but they're not the boss of engineering. So what was your approach when you engaged, for example, with engineering, as you said, even though they're not accountable to you, but you definitely are mutual stakeholders in the security operation? It absolutely is. A lot of my brain time and effort goes to this. First and foremost, you know, I think for me, partnership is the key word. It really drives. It's critical to moving fast and to bringing it to gaining that alignment is to be a really good partner and being a really good partner means building trust. And so, you know, to answer your question, it's really about how can I build trust and credibility with the engineering teams? Even putting security aside for a minute, how do I build trust and credibility with the engineering teams? One of the ways I do that is is by spending time with engineering teams, understanding their understanding their processes. How are they coding? What tools are they using? Today, it's very exciting watching engineers transition to cursor and cloud code, AI assisted code, but, you know, kind of shadowing even and interviewing them, understanding how they do their work is I think that goes a long way, you know, in terms of building trust and credibility, but also then being able to take back and anticipate some of the challenges that might come up when, you know, when we're bringing security into the fold of their workflows. You know, in terms of I think engineering leadership, once again, it's really about translating to the language of other leaders. So for engineering, you know, I think a lot about quality and resilience of code, which are, you know, more universal than, you know, than number of vulnerabilities, right? But the number of vulnerabilities impacts the quality of our code. It impacts our ability to maintain uptime, all, you know, all metrics that our product managers and product engineering think about. And so, you know, a lot of the conversations with peers at that level is about putting it in terms of the, you know, product and engineering metrics. You mentioned artificial intelligence and AI specifically in coding. And I know that this is something, Kristen, that you focus a lot of your time and focus on right now. Tell us a little bit about your background, how you learned about AI and LLMs, and then how you're applying that for, you know, security gaps or risks now. Well, I'm, this is, as many, I'm thrilled and terrified, but I've been thrilled and terrified by machine learning and AI for a long time. Most of my, most of my work in cybersecurity has been around leveraging machine learning. And at one point, leveraging reinforcement learning, which is essentially training an agent to take, to take actions. And LLMs are, you know, of course, a breakthrough, a whole new world of, whole new world of technologies to leverage, but in a lot of ways, the similar technologies. So for me, you know, to the first part, learning about AI, you know, has really been a background in understanding, understanding the different types of algorithms and technologies and, and the cybersecurity needs and figuring out how to apply them. You know, what's so interesting, I think on a lot of CISOs' minds today and business leaders is, we want to use AI, you know, but is AI the right tool, is an LLM the right tool? You know, I think that's a question that we'll continue to see, you know, how that shakes out with where AI finds its home and has the most impact. But today, what that means is we are, of course, rolling out AI at lightning speed in organizations, and we need to do that to, you know, learn and, and get there. But what can we do, you know, in the interim when, you know, as security professionals to give ourselves the most leverage, this is something, you know, I've talked a lot about and think, gave a talk last fall on even, and right now, I, you know, I think a few, you know, kind of just summarize it, and then I can get into some of the details if you'd like. You know, there's two things. One is thinking about how we can set, I'll call it firm guardrails, guardrails that we don't question. And an example of that is access control. We do that with humans. We say, you know, we say Joy has access to A, B, and C, but not D, and she has these, you know, these permissions in our systems. We need to be able to do that with agents and even the sub-agents that they employ. So there, you know, and there's a lot of, a lot of technology built up around that, but there's also very simple ways to do that in your environment today. You know, essentially creating roles or groups just like you would a human for your AIs that have very well-defined permissions. Simple thing. You're already doing it. Now you can, now you're just going to do it for an AI. There's some softer boundaries. That's, you know, kind of the hard boundary category. There's some softer boundaries as well, where we can leverage AI to help us highlight security issues. You know, one of the easy example of that is putting some security rules, such as looking for vulnerabilities into your cursor configuration files so that it's surfacing vulnerabilities to engineers as they're developing, you know, kind of meeting them where they're at. And then the kind of last category, you know, I've alluded to already, it's putting tools and processes in place to learn as, to learn from AI, what the risks are as we go. And so what that looks, you know, I'll give you a very simple example. We set up a weekly, a weekly review process for one of our chatbot LLMs. And we look at what it said, what it did. And we categorize the outputs of AI as high, medium, low risk and high, medium, low aligned with what we would want from a human, right? And so we're quantifying that risk and learning, you know, kind of learning and experimenting as we, you know, as we roll out AI, while still putting those critical sort of hard and soft boundaries in place. Excellent. You know, AI is being used in so many ways. I know, for example, in our products, we leverage it as well. And MITRE Atlas has a great framework for all the different attack methods on LLMs and all the different ways that they can be subverted. But then there's the separate issue of AI that tends to drift or go awry and how hard it is, especially if it drifts early in a use case, how hard it is to re-vector it. So do you think about those things and what's the solution to that? I do. I do. And I think about them probably even more than the, you know, the long term impact of these sort of hidden quality issues, integrity issues that could lead to availability issues. You know, I think that actually kind of alludes to how I think about it. This is why I think incident response and resilience are the ultimate strategy. You know, we are in a time when we, if we accept that AI has these hidden issues and we do not have the ability to, you know, to get in front of them and see them right now, that one of the best strategies we can take is to be ready when those hidden issues do surface. And I'll give you a story. A client I was working with, you know, a client I was working with had rolled out AI quickly and it changed the, was using it to redeploy parts of the website, changed parts of the DNS URLs for, got everything else right, but hallucinated the actual DNS URLs and launched this to production. They took the website down. And this one was sort of immediate, but these are the sort of hidden issues, you know, that would have passed any of our, you know, any of the code testing checks and, you know, it could have sat there for a long time and been deployed at a later point. You know, in this case or in that future case, though, what saves us is having a strong incident response process. That minimized, you know, that allowed us to recover to a trusted state very quickly. Minimized, you know, almost no noticeable difference to the customer on the website, you know, and so it's that we flagged it. We had a process for bringing in the right people quickly. We had ability to, we had flexibility, ability to maneuver our code, and we had basically a backup, right? We had a backup of our code because we were thinking about resilience, not even, you know, even prior to AI, you know, how do we keep our website up when things go bad? So resilience is the ultimate strategy. Yes, it is. Indeed. So if you were advising CISO specifically about AI and security risks, what would be one small step he or she could take now just to look after a top risk in your opinion? Well, AI is certainly a risk. I, you know, I think I'm always a big believer in, I'm going to say education, but I don't mean it as in formally educating all of your employees on security. One of the, I think one of the most impactful sort of education initiatives that I've done is actually just thinking about AI ethics. It was a workshop, a lunch and learn on AI ethics. It's not necessarily about security, but it was, you know, why it was so impactful is it gave a space for people to think about AI ethics. It gave a space for people to really think critically about AI, about, and you could replace AI with any other tool, you know, how people are coding, their SaaS tools, Salesforce. It could be any technology that we're leveraging, you know, and it created a really great conversation, you know, and conversation sticks. That's how people, you know, that's how people really grab on to thinking about how, you know, and not just using the technology blindly. That's the sort of security mindset, the beginning of that same security mindset, you know, and so if you can create engaging conversations about the technology that your company is using, it's an education that sort of scales and continues to grow in people's heads. Yeah, yeah. Communication is key and also the pursuit of those engagements of that interaction with others. And it sounds like a thread throughout your own career progression and leadership is really pursuing those opportunities to align and communicate. Absolutely. It's a very underestimated 10xer in an organization. It is, yes. So I'm curious about what you think about the phrase zero downtime. Of course, we never purport that you will have zero downtime, even if you have good backups, because you have to have the processes in place. You have to have the people properly trained. And let's face it, you know, recovery, post-incident recovery is neither simple nor easy and certainly not fun. But we've learned a lot of lessons about it and we know that security teams themselves have zero downtime. So what does that phrase mean to you? That second part certainly is what first came to mind. I'm not taking a vacation. You know, and I appreciate the, you know, the reality that we may not have zero downtime, but it is the goal. And if I think about, you know, zero downtime, just even a little more broadly, you know, what comes to mind is really that sort of, we'll call it self-inflicted, but security is my chosen profession and it's this contract with, you know, self-inflicted contract with others to deliver, right? Is to keep moving, to be able to keep moving forward, to deliver trust continuously. And so zero downtime, you know, going beyond literal downtime, you know, I think it's really about, it's really about maintaining trust. You know, at the end of the day, I think security, everything we do in security is really about trust, you know, in that sort of contract, any sort of downtime, you know, or unexpected issues can call that into question. And so, you know, it's so everything we can do to not let that very fragile trust with our customers, you know, with our clients, with our vendors, not let that break down. That's really what security is about. Very, very well stated, Kristen. And this wraps up our podcast. So I really appreciate you, Kristen Benaduce, being with me today on this episode of Zero Downtime, and we will look for you on the next episode. Thanks again, Kristen. Such a pleasure. Thanks, Joy. All right. So I'll stop the recording, but if you.
