Transcript
security in art, artfully applying the science to deal with ever shifting technologies, business priorities and threats. In this podcast, we focus on the art of security, seeking out leading voices to challenge traditional approaches to security, identifying components that work, those that might not, and challenging each other to find routes forward that drive the continuous innovation and evolution needed to keep pace with the common enemy. My name is Josh Davis, and in this episode, we're going to consider the art of attack service management and really focus in on vulnerability management. In a world where acronyms for new security solutions pop up every other day, vulnerability management or VM has been a stalwart for security strategies. But what does it take to do VM well? And where can we the industry do better at supporting organizations to reduce risk and shrink exposures through VM? To answer these questions, I'm joined by two titans of the industry, my regular co-host Tyler Reguli, and an exciting special guest, Robert Hansen, aka R-Snake. If you could introduce yourself to the audience, that'd be awesome. Well thanks for that, and you did a great intro there, so that kind of teed it up nicely. My name is Robert R-Snake Hansen, I've been in computer security for 30 years now. Most of that time, I started off as like a browser and web app guy. And over the last, I'd say 10 years or so, I've kind of morphed into more of a network and host security person, mostly on the red teaming side of vulnerability research. But I've also done some blue teaming back when I was at eBay, for instance. Grew up, did quite a bit of consulting, and most recently, we sold a company, Bit Discovery, which is in the external tax service management space, to a publicly traded company called Tenable. And then we started Grossman Ventures as an early stage cybersecurity ventures company, and most recently is RID Evidence, which is what I think we're really here to talk about. And that is a combination of external tax service management and vulnerability management. So that's it. Awesome. Which is why we're here to talk about VM today, or vulnerability management. So Tyler, if you could set us up, set the scene a little bit, do some level setting for people who might not be super familiar with the world of VM. What is vulnerability management? Cool. And Robert, feel free to disagree with anything I'm about to say. I think there's a very big difference, and it's the thing that I want to call out first, between vulnerability management and patch management. There's nothing that bothers me more than when someone says their patch management program is their VM program. I think those are two completely distinct places. So I want to establish that right from the get go. I think the other big thing, when we talk about vulnerability management, is because we have CVE, and we have vulnerability and exposures in there. People like to talk about vulnerabilities and exposures as if they're the same thing. And I think that's another thing that we get wrong a lot of the time. A vulnerability is a vulnerability. A vulnerability could be an exposure. I think it's part of that, you know, superset. But there are plenty of exposures that are not vulnerabilities, things that you might want to be aware of, but they're not things you need to patch or fix immediately. And I think those are two of the places that we really get confused in the VM space. And I think that a lot of businesses have contributed to that confusion over the years with mislabeling products and mislabeling aspects of their VM programs. So I appreciate that. In fact, I have more evidence, hence the name of the company, now to prove that you're right about what you just said than I think I've ever had in my life. I know for a fact that sub 10% of CVEs, which are the exposures you're referring to, there's 800 of them, by the way, but sub 10% of them are actually leading to losses. So the vast majority of them are not actually, for whatever they are, they're not the kinds of things that lead to business loss that we typically think of. That's pretty startling how small that population is. But if you take CVE, let's take my data out of it, because my data is pretty unique, and just use like CISA Kev, for instance, which you can disagree with CISA Kev, and there's problems with CISA Kev. It's around 1300 vulnerabilities, VulnCheck is like 4000 vulnerabilities. So neither or both are right, you know what I mean, like, sorry, neither are right or one of them is right, but the chances of them both being right is zero. So CISA Kev has the advantage of being backed by the government, and, you know, so like a pretty high level of rigor, but it's clearly not the full population of everything that's ever been compromised, because how they actually work is things that have been compromised, for which there is an easy patch, as an example, but that entire population is just 1300 Vulns out of the around just slightly less than 330,000 CVEs, I think it's 329,000 as of today. So that's a really small population, that's a 0.44%, or something like that, really, really tiny population of vulnerabilities that are leading to a breach of any kind. Now, if you expand it to VulnCheck Kev, or some of the bigger Kevs, or I think it's like 6000, that's still just 2% or 4%, you know, depending on of the total CVE population, that's still very small. And so I think one of the problems I have is people do conflate patch management with the vulnerability management program, they're like, we have to patch everything. If a vulnerability management found something, we got to go patch it, because that's the prioritization list is it's on the list at all. Or they'll take CVSS high to low, which basically means you still have to do the whole list, you might be doing it in a very particular order, but if you still have to end up on the whole list or EPSS, but again, you have to get to the whole list. Like, there's never a time when you get to say, I'm done, I've completed this task, unless the task is completed, where you have to do it all. But we know that that doesn't make sense. Because there's like hundreds and hundreds of ways you can attack any individual thing. And they aren't getting attacked. So why? We ran into this company, in fact, many companies now that have over 10 million vulnerabilities and up to hundreds of millions of vulnerabilities, right? I mean, these things are just littered with vulnerabilities, you just kick a rock, and you've hit a vulnerability with these companies, but they've never been compromised. Or if they have been compromised, they've never noticed it's never affected their business, they're still in business. So why does it matter that you have 300,000 or 300 million vulnerabilities or 400 million vulnerabilities? Why does that matter? It turns out it doesn't really correlate with loss, or even breach, which is a lower bar. And so I'm not saying they're free of vulnerability. And those vulnerabilities aren't real things that could be broken, used to break in. That's not what I'm saying. But I am saying that if you're going to patch, clearly, that's not the way to do it. Clearly, that is not having the most impact on breaches and losses. And so that's kind of where I like to draw the line and where the breach and loss informs your patch management, if that makes sense. So I'll just jump in there quickly, because there's a lot of acronyms out there. But I think the one that copped up a lot was KEV, the known exploitable vulnerabilities. And so... Exploited. Exploited, forgive me. Good to be correcting those things. So the issue here is that we're kind of going from a vulnerability database to CVEs that are the 300,000 plus, and then the completionist approach that, hey, if I do all this, I'm completely safe. Whereas actually, that's a lot of wasted manpower and doesn't really accurately measure up to the benefits that you're getting in that risk reduction. So we're firmly on the topic of prioritization here and how you know it with the limited amount of resources that you have, whether that's time or money. No one patches everything. So how do we do a better job? Right there, you said it, no one is patching everything. And so they're naturally creating a cutoff anyway. Like everyone's naturally cutting off somewhere because they just run out of resources or time or whatever. So if that's the case, why not make better decisions about where the cutoff is? That's really all I'm getting to. Like right now you're using CVEs as high to low. Well, or like, let's say the compliance standard says you have to fix everything from CVSS 4.0 or higher, right? That's what the externally for PCI DCS compliance. OK, well, what about the vulnerabilities that are leading to loss that are 2.9? What about the ones that have no CVSS value? There are vulnerabilities that do not have a score at all that are currently leading to loss, about 19 of them or so. And so clearly that is not the right standard, right? The right standard should be what is leading to loss. You know, you could expand the aperture and to say, and we also think you should go down this list, work your way down the list. But right now we don't have the actuarials built into this. And that is really frustrating because that means the adversaries are getting a free pass. They're just able to walk right through our networks because we're just we have no idea that this is even important. We're like not even putting it on the list to prioritize hardly or in some cases not at all because it doesn't have a score to prioritize with. So, yeah, that's pretty frustrating. So now I'm curious, have you taken those 19 or so vulnerabilities and scored them yourself? Because I will say one of my biggest, I guess, pet peeves of CVSS is that everyone wants to argue what the definitive source is. You know, is it is it NVD's fault because there's no CVSS score listed on NVD yet? Is it the vendor's fault because they didn't publish a CVSS score yet? I think decentralized scoring is essentially what it is. But a lot of the time people are not, you know, they're not calculating scores. Now, don't get me wrong. I think CVSS is probably one of the worst ways you could prioritize vulnerabilities. Not just that, the latest guidance coming out from the PCI DSS, from PCI, from First.org, their consumer usage, the one, the first sentence of the conclusion basically says you should not use this for your vulnerability management program, the base score, which is what exactly what everyone's doing. By the way, I got them to put that line into that document. It is just not the right way to do it is basically what it comes down to. Yeah, no, it's, you know, everyone has long said, I think it's it's one of the probably biggest pieces of misinformation that has, you know, existed for the longest period of time, which is that CVSS measures severity. It does not. It never has. It doesn't look at severity properly. And, you know, for a lot of people, it's like, well, you know, it's like, you know, and, you know, forget if we're talking about once you start applying the outside of base scores, maybe, maybe, yes, there are some aspects in there, but everyone's just using the base score. Yeah, everyone just uses again, they go back to, oh, it's on NVD. So it's scored or it's not scored. And that's always going to be the base score that's there, which is is the problem with it. Well, one of my one of the funnier things about this is if you if you look at the European version of CVSS of the NVD, GHSA or whatever it is, they're scored differently. So like, which score am I using? Which one should I use? What about the Russian database? What about the Chinese database? Like they're all scored differently. So that's that's why, you know, it's not real. It's all subjective because these scores don't add up and EPSS doesn't agree with CVSS and Volchek doesn't agree and like none of them agree. And that's that's kind of how I started down this path of realizing I actually didn't expect to find that I expected everyone would converge on a kind of unified answer. And that is not what I found at all. Basically, no one agrees at all, with the exception of the DFIR groups and the cyber insurers. They tend to agree, but no one else agrees. But that makes sense because they're based on actuarial data. They're based on something occurred. They have actual data and they're passing along. They are not adding anything to the data. They're not creating any findings on top of it and using that, that the data is what the data is. And that's it. Everybody else is still, you know, basically guessing. They're using red light stoplight, you know, type security, red, yellow, green or high, medium, low or one through 10 or whatever. It's all the same constraint lists and all of them are failing to understand what the adversary is doing. They've just forgotten the adversary even exists. It's more like theoretically what an adversary might do. So, yeah, what is the adversary actually doing? And I think you used a word that I really like, which is subjective, because that takes it back to the topic of the podcast, which is the art of security. Now, you're arguing an evidence based approach, which obviously is very scientific. But I think there is still a level of subjectivity that goes into deciding because, yes, you can know what's being used in a breach, but there's still a question of what's going to be used in the next breach. And I think there's commonalities that you can look at across the types of vulnerabilities that get used, get exploited. I mean, I'll go with a simple Microsoft one, for example. If you see CLFS, there's a good chance that somebody somewhere has mentioned that as a valid exploit because we've seen a few of them used. And so maybe that has a bit more importance than, you know, something lesser, an office vulnerability that we don't see mentioned as much. It tends to fall into a couple of buckets. There's a lot of drive-by, it's like, you know, inter-explorer type buckets or real player, you know, stuff like that. Like, I have a vulnerable service as I'm going across the Internet, it gets exploited type vulnerabilities, or someone sends me a malicious email and I click on it and it's like a PDF type vulnerability. There's like that group. And then the other group tends to be vulnerabilities that live in security products on the perimeter. Like, those are the two big buckets. I mean, there's a couple of things that are kind of outside that, but those are the big buckets, which is why if you talk with the reinsurers, because they have a sort of macroscopic view of the world, they sort of, they look at all these programs and everything that everyone has. And one of the things they're not concerned about is whether you have a perimeter firewall. It does not impact your security level because that is exactly how the adversaries are getting in, it's through that exact device. And there's a bunch of other things that are not on that list as well, which means that we've been giving bad guidance for a very long time and or our software is just not hardened enough for our guidance to make sense. It's like, well, yes, if firewalls were perfectly secure, then that might be good guidance. But since there seem to be Swiss cheese, not good guidance. And so, again, I think that this all comes down to what the adversaries are actually using. If they're actually using firewalls to break in, we need to really, really rethink the perimeter. If they're using, you know, browsers, we really need to focus more of our energy on that sort of drive-by-download, exploit-payload type stuff. So, you know, I love the evidence-based thing because while, I come from the WebSec world and I'm like near and dear to my heart, but one of the things that the cyber insurers do not care about and they do not measure as a positive or negative about your security program is whether you have a good AppSec program or not. They don't care because there are no losses there. And that breaks my heart a little bit because I spent so much time on that. But what it really means is I think we over-engineered the web application security domain and adversaries had to shift. It was just too expensive for them to be focused there. They weren't getting the return on investment. They had to shift. And where they shifted was very easily deployed, remotely exploitable CVEs. That's where they shifted. And about 40 to 50 percent of all losses come from that alone. And the others come from drive-by-downloads and malicious payloads sent over email or whatever. So if we focus where the adversaries are, we can disrupt them and force them to shift into other areas. But if we don't, then they're going to happily continue to thrive right where they're at. So you've given me two thoughts that are completely different. So I'll address them one at a time. The first one, then, so you mentioned web app security not being sort of a focal point. So how do you feel every time you see the latest WordPress plugin getting headlines everywhere? Because that's one of the biggest things that we see. That leads to breaches, but not loss. There's no losses there. So that's your difference. You're looking, you're focused more on the where the loss actually occurs rather than where. I mean, I think if you want to expand the aperture to breaches, then you're in the kev lists. And, you know, so then you have VulnCheck, which does have tons of those vulnerabilities you just described. And by the way, funny enough, that still doesn't count because those are considered to be like HOTS software, not custom code. And so all the custom code stuff, still no losses and not even not even breaches, literally nothing. If you want to explain the aperture to kevs, then yes, there are exploitation happening, but no losses. And the amount of things that lead to loss is very, very small. And Robert, I'm really interested in that because I come from the incident response side of the world. And, you know, knowing that attacks aren't really one and done and they're often chains of lots of different vulnerabilities, which I think is where that conversation comes in. Once you get in, the amount of CVEs expands significantly. So I just want to know how well the insurance brokers, I mean, DFI teams, I see it as insurance brokers. I'm not so sure about, I don't know that world as well myself, but how well do they connect a vulnerability that's maybe 10 steps downstream in an attack path or attack sequence? How well are they able to do that and tie it to loss? They're entirely actuarial based. So if the DFIR guys don't track it, they don't get it. And so I'll give you the stats I have, which I don't know if we're right or not, but here's the stats I have. So one vendor said that they can get the initial access vector and 60% of the time, which I realize is not quite your question, but I'll get there in a second. The second one said 50%, which I think that second guy knows slightly better than the first guy. And the last one I think knows the best out of all of them. And he said, if you remove ransomware, because ransomware is super easy to find for a variety of reasons, they tend to use the same exploits over and over, then they can only find the initial access vector in 10% of the cases. And there's a wide variety of incentives that are messed up and technology that isn't there. And frankly, just logs don't exist in many cases and et cetera, et cetera. That means that you're just not going to find it. And so you're right to be suspicious that the insurance carriers are not getting enough data. So we need to work more with the DFIR teams to help enable them and fix those incentives, fix the software so they get better actuarial data. Very cool. Now I'll shift to my other question, which is when we talk about the perimeter and talk about the security devices that are on there, how much do you consider or how important, I guess, do you consider configuration, system hardening of those devices? Because I think one of the things I commonly notice when we're talking about, you know, we see the latest firewall breach or the latest VPN concentrator being used time and time again. A lot of the time those come down to management interfaces that have been exposed externally. And anybody who's following basic system hardening going back 30 plus years knows you don't put those interfaces on the Internet. So what role does that play? I think that's roughly 100 percent of the 40 to 50 percent perimeter that are outside. Roughly 100 percent. Not exactly, but close enough that you could pretty much make a hard line. If you want to expand it slightly and talk about authentication as opposed to just having the web application console public, then that even just that one thing accounts for a huge amount of reduction in loss. So you can still have it open. But if you have MFA, there is a massive reduction in losses. And the reason we know that is because one of the cyber insurers, for competitive reasons, decided to test removing MFA from their criteria to see what happened, if they could make more premiums that way. And they did, but they also had massive losses because no one turned on MFA for those external devices. And all of a sudden, you know, there's breakthrough claims all over the place. So they had to reinstate that. So we know that MFA has a huge impact on losses. But yes, if you could turn off completely and have no external access, you win. Right. You're not going to break in. There's MFA or no MFA, you're not breaking in that way. Cool. MFA is always an interesting one, because I wonder how many people deal with MFA fatigue with the number of places that it exists. Yeah, but we're talking about something very tiny and super critical. It's not like a consumer level thing. No, I agree completely. It's not like your Facebook account or something, you know. Yes. No, a hundred percent. I agree in those places that it just makes sense. But it's just one of those things that I always think about because people always post complaining. Right. So it doesn't really apply. If you have a better mousetrap, I'm sure people would love it. Right. One day I'll invent that and then I'll be rich. But yeah, no, it is a good point that those are important places to have MFA enabled. Great. So I think there's a lot of focus on the prioritization, how things can do better. So we've got maybe about five or a little bit over time to kind of come to the end of this. So how can we look at what we've explored so far? How can we kind of evolve vulnerability management, whether that's in the tooling itself or whether that's in how businesses and organizations use it to be successful? Because I hear the problems, but I'm not convinced that it's something as that example you gave, you should just turn off and then turn entirely to other methods. I think it still has a place, even if my position as a former SOC analyst would be to log on to a host. I'm scratching my head and just look at the CVs, the listed decks. It helps me understand the technology and where I might look next. So what does that world look like? I think that I have no idea because I'm not inside the head of every single VM partner who's going to want to use this software for whatever reason they're going to want to use it. Some are just going to want to double check their work. Some are going to want to prioritize on top of their prioritization system. Some are going to see, hey, should we change our policies? And to what? You know, people are going to read this in different ways. So this is a major shift, I think, for our company that I haven't heard anyone else talking about. I don't want to be in the business of prioritization. I just want to be in the business of saying, here's what we think is true. We have a warranty if we get it wrong and you're on your own because you were always on your own. But here's all the evidence to make a good decision. Here's everything we know about this vulnerability from all the places, like all those crazy databases that are very hard, by the way, if you've not actually tried to pull them down, it's incredibly difficult. But we've pulled them all down. We parsed them all for you. We can tell you what everyone says in every language about everything, everything about this vulnerability. We can also tell you stuff that is not public, like really intense behind the scenes, what the insurers are saying about it, what DFIR are saying about it, whatever. And from there, we'll give you what we think is the right choice of prioritization. But if you know that Project Bob is super critical to your company for whatever reason, Project Bob is super critical. Well, why am I going to get in the way of that? Like, as long as I give you the tooling to make your own decisions and basically back up the decision you are going to make anyway with data, with real data, I think people are going to be happy. Now, would I build it the way they would build it? Probably not. But that's OK. Like, I'm not going to tell them, hey, all your policies are wrong and you did it bad. I'm going to say, well, based on what you want to do and what your company says your goals are, this is the controls you're going to need to write to make those decisions. And then they'll be happy because that was always what they wanted to do in the first place, because at the end of the day, this really is a subjective thing. But it doesn't have to be if they choose to use the actuarial data. They can make the draconian choice and really do it the right way. But a lot of people are not going to do that for a variety of technical or business or compliance reasons or whatever. They're just going to say, no, we got to do this other way. Or a vendor that relies on us says we have to do something. And so we're what even though we know the right thing, they don't and we're beholden to them. OK, that's a business choice. You're going to lose millions of dollars if you don't do it the way that they want to do it. That's obviously the right choice for you. If I stay out of the way, I think I win. If I am trying to tell them high, medium and low and whatever, like, I think that's where everyone gets into trouble. Yeah, those those kind of mandates that are not focusing your limited power and resources as effectively as possible really, really bug me and irk me. And I think in VM, there's a lot of times where people feel like they're busy, they're doing work and they're making the list go down. And oh, tomorrow there's a bunch more. So it's almost that I'm secure boss because I've been doing this and working really, really hard rather than trying to be more intelligent. So it's not just about prioritization of those vulnerabilities, but it sounds like this is a bigger conversation about how do you find that subjective focus for your security strategy? But Tyler, where where have you where are you still where you started before this conversation? Have you kind of shifted anywhere at all? Where are you at with the VM and its place in security? I think I'm in the exact same place I've always been. I think it's it's super critical. I think that VM is always going to be one of those basic levels of cyber hygiene. I think Robert's made some really good points about where we look and what's being used. I think it's an interesting way to look at presenting the data. But I'm always going to say, hey, vulnerability management's important. I'm always going to say I prefer remote or network based vulnerability management over the host based stuff and even more so over some of the other fake VM that we see out there. I'm always going to hate the concept of potential vulnerabilities. And I'm always going to hate CPE based vulnerability detection because that's not how you determine if something's vulnerable because it happens to be a specific piece of software. So I don't think my mind has changed at all. I mean, I've known Robert for a long time. So I would have been surprised if I had changed your mind, because I think what I'm saying makes sense to everybody. It's intuitively how you've always thought about it. It's just that we never had the data before. And so now that we have the data, you're like, yeah, because that's how we should have always been doing it. Right. Yes. You know, I don't get a lot of disagreements on phone calls. I get a lot of head nodding, maybe some head scratching, like, how are you doing it? Or like, how does that work in my environment or whatever? But not not fundamental. Not like this whole concept is bad. More like, OK, like, how do I how can we make it better? Like, have you have you talked to this team or, you know, like they want to improve it. They don't want to tear it down. Or, you know, if they do want to tear it down, they're quickly confronted with the actual data underneath the hood because they're like, no, EPSS is the way to go. I'm like, OK, let me show you how EPSS compares to this data. And I'll do a side by side and show you everything that's wrong with it. And and how like there's current exploits right now being used and it's EPSS two or something or point two rather. And so it's like, yes, it's I understand the concept of wanting to predict where the adversaries would go. It turns out mathematically that is incredibly, incredibly unlikely, like really, really, really unlikely. And so much so I don't I would not bet on EPSS. Not that I'm competing with EPSS, but I'm saying, like, if you're going to put all your your apples into one prioritization bucket, I think there's a better way to do it. And EPSS is the standard that tries to predict how likely it is to be exploited in the future. Right. But within the next 30 days. Yeah, it's a it's a shifting 30 day window. No, I think the approach you're taking reminds me of when one of the things I was doing with VM customers was consulting with them and telling them, you know, don't worry about what the CVSS score is. Don't worry about what our scoring says. Here's what you should fix. And going in and working with them based on their environment and based on, you know, what was actually at risk at that time and the results of doing that kind of, you know, collaborative work where you're actually looking at things proves to be very valuable. And so when you're taking that that straight shot at what exists and what doesn't exist and going on that approach, I think you can't beat it. I agree. So I think that that kind of collaborative message that ultimately this is going to have to be your own context that in business context and asset context, you have to add to this to make your decisions and how to focus. But using some of the stuff you talked about today in the direction of travel, vulnerability management could do a little bit better at giving you more information to make a better informed decision when it comes to your patching strategies. And if I would go so far as substantially better, I'll upgrade to substantially. That's absolutely. I think just it's a real example of some often people say that, you know, security hasn't changed because we still do patching management. And since we had since computers needed it in the 90s, but really it's the adversaries that have changed. It's not known for vandalism anymore. It's that real tie to to loss and the explosion or acceleration of their capabilities. So I think that linking it to loss was probably the biggest takeaway that I'll I'll continue to use as I go forward as a security professional. But thank you very much for your insights. And is there any anything you want to kind of plug or highlight to people where they can find you or something they should go and look at that you've been working on? Well, I'm certainly posting a lot of LinkedIn if you want to check me out there. But frankly, if there's anybody listening and this is like tickling them a little bit and they want to know more, just reach out. We have a design partner link somewhere on the web page, rootevidence.com. We're going to need an army of intelligent security people who at least want to make this better. You know, they're not happy with things, how things are going. And so if that describes you even a little bit, like, please reach out and we'll we'll start talking to you. Call to arms. I love it. So thank you very much. That's all we've got time for in this episode of Art of Security. People listening, wherever you're finding us, you can engage with us. We do love to hear your opinions on this because, as you said, the art is subjective. Which leads to some really interesting conversations that make us all a little bit more battle ready, let's say, when it comes to deploying our security strategies. So thank you all and we'll see you next time.
