Transcript
Nist and I lead the BigID University team and I'm really excited to be here today with Heather Kuhn, who's going to talk a little bit to us about the concept of responsible AI. And when we get into this topic, we're really talking about what can we do to ensure ethical, safe, and accountable AI use across the organization. Heather comes to us from a very unique perspective in that she's our senior privacy counsel at BigID. And so Heather, why don't you tell us a little bit more about yourself and your background? Sure. So, so my role here is a little bit unique where I have kind of three prongs. I have the part where I actually run BigID's own product. So I am customer zero here where I live and breathe the product every day. I also work with our product team to help advise on what the tools should look like. So we ensure that kind of our product timelines and our regulatory timelines stay in place. And then thirdly, kind of go out and make sure that I'm talking with customers, that I'm staying part of the, the environment. So we understand that BigID is really part of the thought leadership around what's happening with many topics, including responsible AI. I also, I teach at Georgia State College of Law here in Atlanta, Georgia. And so I am a professor of cybersecurity law as well as privacy law. And one thing about me, I'm actually a sociology major undergrad. So I am a non-technical person here, but I've always really loved kind of being in the tech space. You can just ask my husband or kids, they'll, they'll agree to that. And so it's really awesome to be able to kind of always be playing with this. And I think that that really helps bring a perspective of understanding how these things actually work and not just kind of the policy documents that exist to describe them. Very cool. So what's a fun piece of technology that you've been digging into lately? So in my household, you'll hear a lot about using the different chatbots. My eight year old, she just did meal planning to help us figure out all of our, our grocery shopping and party planning needs. And so our college students are using it for different things in school. So I think we're, we're always really trying to, to take that latest and greatest and figure out how can we responsibly use it in my personal life as well as now my professional life. I love that. Okay, great. So Heather, thanks for giving us some of your time today. So let's start with some of the basics on this topic. So responsible AI, it sounds to me like very broad, could potentially be loosely defined. When you hear that term, can you tell us what that kind of means to you at a high level? Sure. So, so first off, obviously this is the buzzword that you're hearing everywhere. And so I think it is really important that you start off with a common understanding of what it actually means. So for me, when I think about responsible AI, I really think about it kind of at the accountability. So at every layer and not just the output. So think about it all the way down to the data that's going to feed your model, which is really kind of the foundational level that you need in order to understand what is responsible AI. So I think that you see a lot of organizations have some type of responsible AI policy that probably lives in a PDF somewhere that someone created when this first started, but that's not really a program. That's kind of a placeholder. And so I think that responsible AI is really taking that next step. So it means you actually know what is your data, what systems are they touching? Can you demonstrate what the outputs are? Can you make sure that it's not going to expose personal information or regulated information? Do you have a mechanism in place to audit what's going to happen and when it's going to happen? And I think that's really the difference between saying, you know, we're committed to ethical AI, which is something, you know, you might put on a website, but actually being able to prove that out. And I think, I really think about it kind of in terms of trust. So responsible AI, it talks about, you know, earning that trust from not only your employees, but your customers, and increasingly the regulators. You can't really earn that trust with just a policy statement. You have to do it through action. So you're going to see controls, visibility, accountability, all of those things come into place when you're thinking about what really is responsible AI. Okay, that's really helpful. So takeaways there, it's a program with really core foundational pillars that you need to think about rather than just trying to check the box real quick and easy so that you can say you have responsible AI. So let's dig into that a little bit more. What are some of those foundational components that you would need in a proper responsible AI program? So for me, I think that typically it's going to have about four layers. So you have, first off, your data governance piece. And this is something we've been doing. You do know what your data is, what systems is it flowing into, is it personal data, is it regulated, is it sensitive? If you can't answer that, you can't operate responsibility, and that's kind of full stop. Second would be your access controls. So not every model needs access to everything. Your data scientists, your marketing teams might object to this, but from a responsible AI perspective, this is an important one. You want to approach it through a zero trust lens. So AI systems should have the minimum necessary access to do their jobs, and that access should be governed, it needs to be logged, and it needs to be reviewable. I think the next part kind of going along that reviewable piece is the audit trails. Accountability requires receipts. So if an AI system has made a decision or it surfaced data that it shouldn't have, or it was used in a way that maybe created risk, you have to be able to go back and reconstruct what happened. And that's not just good governance, that's really a legal requirement. Sometimes that can be difficult because we see AI as almost this black box where we don't necessarily know what's happening, and that's sometimes where the legal requirement and the innovation part butt heads because you do have to have that visibility into it to make a truly responsible program. And then I think finally is just that alignment with the legal and regulatory standards. And fully acknowledge that right now it's a bit of a moving target, but responsible AI doesn't exist in a vacuum. It exists in a regulatory environment that is evolving very quickly, and you have to have a program that can be built to flex with that. And I think the important thing to emphasize is that these aren't necessarily sequential steps. They have to work together. So data governance without audit trails is going to be incomplete. Access controls without visibility into what data is being accessed is going to be meaningless. But also emphasize that these aren't new, right? These are the same muscles that we've had to exercise when it came to new privacy laws, security laws, and now we're kind of taking that and putting it into the AI lens. And tackling, yes, there are some new novel issues when it comes to AI, but these are skills that, especially here at BigID, right? We've been working on this for 10 years. This is something that's ingrained in the fabric of what our product is, and I think that that's an important thing to remember. This is something that people have been contemplating and trying to solve for a long time. Right. Excellent. Thank you. That's really helpful information. So you hit on, at the end of your discussion there on some of those core components of a program, the legal aspect, the laws around AI. And I think it's really interesting to get a chance to speak with you because you have a really unique background, right? You've got a ton of domain expertise in data and AI security, privacy, and compliance, but you're also a legal professional, right? You teach at law school. You're a lawyer yourself. So tell us a little bit about how you see the world through your perspective. For example, if we're going to implement a proper responsible AI program, what's in it for the business? So I think from a legal perspective, the risk calculus has fundamentally shifted. So for a long time, companies could kind of treat AI governance as aspirational. It was a value statement. It was something on a website. It was not necessarily a compliance obligation, but I think that that's fast changing and that that window is closing. So for example, we now have the EU AI Act, which is the most comprehensive AI-specific legislation in the world. It's already in phased enforcement. In the U.S. here, we have sector-specific guidance from the FTC, from the EEOC, from the CFPB, all of which are looking how AI intersects with things like discrimination, consumer protection, financial services. So kind of all of these existing industries, you're seeing kind of this AI layer put on top of it. And then we also have state-level AI laws, which are starting to mature, particularly around automated decision-making, which again is something that's a little bit more contemplated in the privacy realm. And so they've just been building on that when it comes to AI. And so I think really the first reason here that you want to make sure you're thinking about this is that you have legal exposure. So if you're using AI irresponsibly, you're almost inevitably going to be using, you're in violation of something. Even if no one's knocked on your door, you're probably violating it if you're doing it irresponsibly. So second is going to be reputational risk. And I think this is going to be probably the faster moving one since that type of impact is going to happen faster than maybe a regulator can get to you, is that organizations are going to kind of weather this next wave of scrutiny. The ones that are going to get through this are the ones who can demonstrate and not just, again, have it on their website, that they have programs that are responsible. So customers are asking about it. Employees are asking about it. Boards are asking about it. So you have kind of all of these stakeholders that are at the table that are questioning what you're doing that might be moving faster than some of the regulators. And I think the third reason, and sometimes this gets overlooked, is really business resilience. So when you have the governance infrastructure in place, you can actually move faster with AI adoption. So rather than seeing this as kind of a blocker, as something that's going to stop innovation, you're not, you're able to be more responsive because you're not constantly retrofitting controls in after a crisis. You're not waiting until something has gone wrong in order to fix it until a fine has been imposed. You're saying, no, we're going to be proactive about this so that when it happens, we can actually move faster. Yeah. And I think that's a really important point, right? Because businesses are under so much pressure to move quickly. And the field of AI is changing so rapidly. Like, if you're not moving quickly, you're actually really falling behind. Right. So I love that having this program in place is actually a catalyst to enabling you to properly move forward at speed. So one of the things you hit on there in that discussion was a reference to the AI legislation that currently exists, specifically the EU AI Act. So I've heard a lot about that, but I'll admit, I don't know a lot of the details or why it's important and what we should consider. And I'm sure there are people in our audience here that are in that same position. So can you tell us a little bit more, what do we absolutely need to know and understand about that EU AI Act? Sure. So the EU AI Act is a risk-based framework, which sounds straightforward until you realize that the classification determines everything. So this is where your obligations, your timelines, and your potential fines are kind of all drawn from. And the key thing that people need to understand is that the risk tier structure. So there are prohibitive practices, things like social scoring or real-time biometric surveillance in public places that you just can't do. They're completely off the table. Then you have things like high-risk AI systems, which cover areas like hiring, credit decisions, education, law enforcement, critical infrastructure. And in these cases, if your system falls within that category, you're looking at mandatory requirements. So you have things like assessments, transparency requirements, human oversight requirements, documentation obligations. So there's a lot more kind of guardrails that are put around these high-risk categories. And one thing that you hear is a lot of people, you know, kind of hear EU regulation and they might say, hey, it doesn't apply to me. I'm not in Europe. I'm all good. But this is actually a mistake. So the AI systems that affect people in the EU are anything that touches someone who's there. So that could be an employee and now you're in scope. So it's not simply I have a physical presence with an office in Europe. It's going to be, do I target people that are in the EU? Do I have employees that are in the EU? So that that scope of who it impacts is a lot broader. And one of the things that I, you know, tell customers, tell my students is that you need to start with what your data is and where your data is. So that classification piece, because understanding where your AI systems fall in the risk hierarchy is going to determine everything else. And so you also want to make sure that you don't assume that your vendors have done this for you, because it's something that you ultimately are going to be responsible for. So you need to kind of flow down the entire pathway to understand what is your obligation. It places a significant obligation on the deployers, which are the people using it, not just the developers of the technology. It has also phased timelines. So some of those are already in place. So like the prohibited practices that we were talking about, that already went into effect in February of last year. The high, high risk systems requirements start rolling out between this year and next year. So taking an approach of like, we'll deal with that later. That's in the future. We're kind of past that. We have to start worrying about it. Interesting. Okay, fantastic. So that was really informative on what's going on on this front in the EU. But you and I, we're both from the United States, for example. So can you tell us a little bit about what's happening on the legislative front in the United States or in other parts of the world? Yeah. So the US is taking a very different approach than the EU. And honestly, you know, it can be an opportunity and a challenge for organizations trying to plan ahead. So at the federal level, we don't have a comprehensive AI law yet. What we do have is a patchwork of, you've got agency guidance and existing laws being applied in the AI context. So like we were talking about earlier, kind of across the board and all the existing industries, you're seeing this kind of AI piece put on top of it. So for example, the FTC has been very active. They've made it clear that, you know, existing consumer protection and anti-discrimination laws apply to AI. They've taken enforcement action accordingly. Same thing in the employment context. The EEOC has issued guidance on AI in hiring. Various banking regulators have also weighed in when it comes to AI and credit decisions. And at the state level, it's moving faster. So Colorado passed an AI law focused on high-risk systems and algorithmic discrimination. You've got California. They have the CCPA, which many of us are very familiar with, which creates rights around the automated decision-making. The regulator in California, CalPrivacy, they have been developing additional regulations on AI and automated decision-making that are expected to take effect next year. You've got Texas, Illinois, other laws that are all touching AI in specific contexts, like biometrics and employment. So it's a little bit more piecemeal in the U.S. of kind of industry and use case specific. And unfortunately, or fortunately, what this creates is this kind of very complex compliance mosaic, which sometimes, you know, can be even harder than just a single comprehensive law. So like in the EU, you've got the one law that covers everything. In the U.S. here, you're looking at a bunch of different things. And so the complexity is created because you have to track all these jurisdictions, all of these triggers, all these timelines, kind of all at the same time. And so my advice is, you know, don't wait for that federal clarity. Build your programs against kind of some of the most stringent applicable standards, and you'll be better positioned regardless of what comes next. Build out kind of on best practices now so that you're defensible. I know I, as a legal professional, I don't want to sit before a regulator and say, we haven't even thought about it. We're just kicking the can down the road. You know, I wouldn't be able to go to them and say, we didn't know what your rules were, but we have been very intentional, and here's the plan that we used, and here's why we used it. And that's going to be a much better position to put your company in than to say, well, you didn't tell us yet, so we just have been waiting here, which again, not the position I want to be in as the legal professional sitting across the table from a regulator. I agree. Preparedness is the key. So fascinating difference between what's going on in the EU and what's happening here in the U.S., but given the rapidly evolving landscape around AI legislation, you know, you hit on the nail on the head saying you can't just do nothing until there's clarity. Preparedness is key. So if you could give people advice, like what should they focus on right now to try to ensure compliance given what we know right now, but also thinking a little bit towards the future, what would you recommend? So the things that I'd want to prioritize is, first off, inventory all of your AI systems. You can't govern what you don't know about. So that means, you know, your sanctioned systems, your third-party tools that have embedded capabilities, and the shadow AI that everyone's talking about that employees are using, you have to get that visibility first. So shine the light, understand what's going on, and this is really going to be foundational to everything else. Once you know where things are, then you can take the next step, which is going to be classifying that data. So this is, you know, of course going to go hand-in-hand with your inventory is understanding for every system that you're running, you need to know, you know, what data is it accessing, what category that data falls into, so things like personal, sensitive, regulated, and then you need to know whether the appropriate, whether that's appropriate given the purpose. So watching out for kind of secondary purposes or creeps, scope creep, are important. Responsible AI is really fundamentally a data governance problem. And then thirdly, I think about documentation habits. So every major AI regulatory framework requires that you be able to demonstrate that governance practices exist and not just describe them. So that means your policies, your assessments, your audit logs, your incident response, your incident records, you need to be able to produce all of those things if someone asks. So it's no longer just a pinky promise that you're doing it. It becomes a show me your work and tell me exactly what you've done. And the companies that are ahead of this aren't necessarily the ones with the biggest compliance teams. They're the ones that, you know, connected their AI governance to their existing governance, data governance infrastructure. So use what we already have created as much as possible. Use those tools that you've already found that can be leveraged to solve the problems that exist in today's world. So that's where, again, platforms like BigID really create the leverage because things like the data, the classification, the access visibility, they already work. They're already there. And it's not separate from AI governance. It is the AI governance. Yeah, that makes sense. So let's talk then a little bit about all the many various sources of AI that one might find in any given organization. For example, you've got those sanctioned AI systems that have kind of the stamp of approval, typically from your IT and your data security teams. But, you know, in my experience, I'm going through right now renewing a handful of vendor applications that my team uses. And almost every single one of those has AI features and capabilities embedded into them. So how do you approach evaluating your vendors and the potential risk that comes from third party AI? So I think this is one of the most underrated risks right now is kind of this third party problem. So every SaaS tool, every kind of one that your employees are using. So think about your collaboration platforms, your CRMs, your HR systems. They have some type of AI baked into them. And most of those tools were procured before, you know, the AI functionality existed. These have been baked into your infrastructure for years. And so it's a massive kind of retrospective review problem of where do I even start? And the way that I approach vendor AI risk is through a few key questions. So what data is the vendor's AI processing and under what legal basis? So is that data being used to train models? Which models? Is it a vendor or third party? You know, what happens to that data after the AI interaction? And then critically, what does the contract actually say? So kind of looking at holistically what our interaction with this vendor is and how that kind of plays out is really important. And that allows me to prioritize what I'm going to look at first. A lot of the standard vendor agreements, again, were written before generative AI was really a meaningful consideration. And so you're seeing data processing addendums that exist, which is good. We're happy there. But they don't necessarily address some of the nuance or the technicality around AI training or some of the novel issues that come with AI. They're just thinking about it more broadly on the data perspective. You're also seeing that subprocessor lists don't always reflect LLM providers. And that's another gap that needs to be closed. I think about this, again, through that idea of zero trust that I mentioned earlier, is that you wouldn't give a third party system unrestricted access to your network. You shouldn't give a third party AI unrestricted access to your sensitive data. So those kind of, again, go hand in hand. So the principles of minimum security access are going to apply here in the same way. Practically speaking, I would say this really means that you, again, need to work with other stakeholders, work with your procurement teams, work with your legal teams to build AI-specific questions into existing processes. Let's leverage systems that already exist, leverage the operations that already exist, and just kind of close those gaps to look back retroactively on what you might need to address in this new AI context. All right. Thank you. That's very helpful. So talking about AI, obviously, one of the things that stands out to me is just how fast it changes and evolves. Really, when I think back over just the last two years of my career, AI was just starting to be talked about quite a bit, and now it's everywhere. It's ubiquitous across the things that we do on a daily basis. Now, I know it's hard to do, but let's imagine that you've got a crystal ball. How do you, Heather, how do you think responsible AI might evolve over the next few years? And what should we be thinking about, right? What should we think about to try to stay ahead? It's standard issue for all technology lawyers to get a crystal ball, so don't worry. Okay, that comes when you need a baby, right? But I think what we're going to see is responsible AI shift from kind of this voluntary aspiration or best practice to a verifiable standard, and that shift is going to happen faster than I think most organizations expect. Kind of as you mentioned, these AI chatbots really only became mainstream at the end of 2022, which is not that long ago, right? So within the past three and a half years, we have moved rapidly into the adoption stage of a technology we're still figuring out. And so a lot of this kind of AI governance exercise is really kind of that building the airplane while flying type of situation. And so I think on the regulatory side, the EU AI Act is going to generate case law. It's going to create enforcement guidance that's going to ripple globally in a similar way that GDPR did, where they're kind of setting the standard, they're helping us understand it, they're teasing it out for us through case law and guidance. And companies that treated GDPR as a European problem and then kind of scrambled when California did it are going to have a problem. And I think we're going to see a replay of that within AI regulation. If you've taken the time to kind of get yourself in order when it comes to EU AI Act, you're probably going to be in a much better place right now as the rest of the world figures it out. On the technology side, I expect that we see more demand for what's called AI providence. And that's kind of the next wave of maturity. And that's going to be the ability to trace not just what an AI system did, but actually what data it was trained on. So again, breaking open that black box, understanding what data it accessed, what decisions it made, how that was influenced. And that's a data problem really at its core. And organizations that have strong data governance infrastructure are going to have a significant advantage here. In terms of roles, I think every organization is going to need someone who sits at that intersection between these teams. Having that cross party team, that cross functional team is going to be very important, especially at that intersection between legal data and AI operations. So whether that's having a dedicated AI governance function, which some organizations are building out, or whether kind of it gets absorbed into maybe privacy, security teams, it's going to depend on the organization, but the competency has to exist somewhere. And frankly, I think that organizations that treat responsible AI as a competitive differentiator and not just a compliance checkbox are going to be the ones that build the most durable trust with their customers and employees. And that's where the opportunity is. Take this as a way to enable business to drive forward, to move faster, as we were saying earlier, by having those compliance structures in place rather than an obstacle that needs to be overcome. Yeah, very good. All right. Well, Heather, I can't thank you enough for giving us some of your time today. This is a fascinating topic and I know I learned a lot, so I'm really hoping our audience did as well. So just to recap a little here, we talked about a lot of things around this theme of responsible AI, starting with the four pillars that make up a proper responsible AI program, which goes far beyond just simply having some sort of policy statement or marketing material on your website. We talked a little bit about the business value that comes from building such a program and that it allows you to responsibly accelerate your adoption and usage of AI technology. We talked a little bit about some of the fast-changing regulatory landscapes, specifically the EU AI Act and some of the more statewide things that are happening inside of the United States. We talked about the whole avenue of third-party SaaS technology in your organization and how all of those solutions are implementing AI and how do you deal with that from a responsible vendor management perspective. And so really, really good information. And so with that, we're going to wrap things up here today. And again, thank you, Heather. Appreciate your time. Thank you.
