Transcript
I don't think they tell you what you think they're telling. Our guest is Wade Baker, co-founder of Scientia Institute, here to break down the metrics cybersecurity relies on and where they fail. We have a problem of what are we trying to do in security? What is a good security program? What does it mean to be performing? Just because you don't have an incident doesn't mean you have a good security program. And just because you have some incidents doesn't mean you don't have a good security program. But the just measure something without the here's what you really ought to be measuring, I think is problematic. If you ask a lot of people, how much did your breach cost and how many records did you lose? It assumes that there's a linear relationship. But when you do that math, you get these astronomical numbers in large breaches. I'd say that's the number one reason why this metric won't die, is because when you do the math and you multiply it by, hey, we could lose 100 million records. That's a really large number that helps justify a lot of security spending. Are we measuring the right things, or in fact, just the things that are easy to measure? I'm Ash Hunt, and this is The Watchtower. Wade, great to have you. I appreciate it. I'm looking forward to the chat and always happy to talk metrics with measurement-minded folks. Yeah, I know we've had lots of discussions on this, so I'm super excited to get down into some of the detail. Maybe just as a bit of backdrop, I feel we've built an entire industry around metrics that feel like they're right. But certainly from my experience, and I'm sure you've had the same observations, they typically don't reflect how risk actually behaves. So I guess just as a starter for 10, what are some of the most common metrics that you see that are completely misunderstood in the industry? Yeah, there's a good list of them. Top of mind, mean time to remediation or resolution, whatever your R is, is a big one. I'm sure we'll talk about why. It's not a terrible metric, but a lot of people don't understand what it's actually measuring and not measuring. I often get frustrated with the cost of a data breach on a per data record basis. I've been trying to kind of put the nail in that coffin for a long time, but it just won't die. So that's another one. And then there's metrics that we use to, say, measure, do we have a good security program or not? And whether that's, oh, we patched X number of vulnerabilities or, you know, we had this many attacks or incidents or whatever. I think those are, again, not bad things to measure, but I don't think they tell you what you think they're telling you. A hundred percent. And I, you know, obviously as an XE, so I've got a lot of subjective views. And I think it's probably quite an opinion-based thought for most of the industry in terms of what they deem as successful measurement criteria for their own program efficacy. But like you said, before we get into some of those metrics and stuff, I kind of wanted to start just getting your thoughts on really the psychology behind it. I, like you and I have both said at the start of this conversation, and we've said it before, and I know that there are those in the industry that share our views too, that for some reason, we're still hamstrung by pretty poor approaches to measurement. And it feels like after decades that we've still not made the type of progress that perhaps those of us in this side of the fence kind of want to see. Why do you think it is when you spend years looking at particularly data in significant scale, why do you think it is that we're still not getting this right? When there's lots of the right thinking, lots of the right examples already in place to measure stuff far more effectively than we are? Yeah, I think there's a combination of reasons. And I know that's an easy way out of this answer, but I'll just list a few and I'm sure you can resonate with a lot of these. I think, at least over my career, which kind of mimics the growth of the cybersecurity industry as a whole in that timeframe, it's gone from infancy to where it's a full-fledged industry now. And over that time, I think there's been a realization of, hey, we got to measure something, right? People are asking us to track things and prove that what we're doing from a security perspective has results and is beneficial. And so I think that that pressure is part of the problem. I mean, it's a good pressure. We need to measure things, but the just measure something without the, here's what you really ought to be measuring, I think is problematic, right? It sort of creates a drive to, well, I can measure this and I'll start reporting this. I'm not sure what it tells me, but hey, it's easy to measure and I've been reporting it for a while and people expect it now. So we got to keep doing that. So I think that is part of the problem. I also think it's part, we have a problem of what are we trying to do in security? You know, what is a good security program? What does it mean to be performing well? If you jump out of security for a moment and you go to business metrics or sales metrics, like they know what measures indicate a healthy, profitable business that is growing versus one that's dying. And they know what a good salesperson should look like from a metrics hitting their numbers perspective versus not. And I'm not sure everyone agrees what that is in security. Is it not having any incidents? I don't think so because incidents are sparse. You know, just because you don't have an incident doesn't mean you have a good security program. And just because you have some incidents doesn't mean you don't have a good security program. So there's questions like that, that I think are legitimate. What does good look like? And therefore, how do we measure that appropriately? And I could go on, but I feel like that's kind of a long talking segment there. So I don't know. What about from your perspective? Because you've managed companies. I'm a measurer and to a lesser extent, a modeler. So I just I go around and count things all the time. You've managed security companies and programs. So I don't know your thoughts. You know, I've tried to bridge the two together because, I mean, the number of roles that I went into and you would you would inherit all of these compliance based reports. And you're like, what on earth are these? You know, is the objective behind these? And I was explaining to someone the other day, and this is actually maybe even a bit more controversial, that it was in the context of why I think typical quant programs struggle to scale. And I said, actually, one of the techniques that I use, which I think feeds into your point around the objectives behind good measurement is I said, we simply started tracking like quotidian activity in the business, like day to day, every event management in the business. We simply did an observation period where we started to look at what was going on, like what stuff was taking up people's time, what stuff was hitting people's inboxes day in, day out that it was causing us bleed, as I would call it. And we just started building decision engineering around those things first before we looked at anything else. I think that concept is pretty tricky psychologically for the security industry to get behind, because I liken it to the sniper in the general. I think a lot of security practitioners, analysts and engineers think like snipers. They have problems that they fixate on. And their goal is to simply eradicate that problem. Right. And rightly so. I think CISOs and the CXO in technology think like generals, or at least they should, which is we have objectives. We need to get over the hill. We recognize that 80% of it's going to be achieved. We're probably going to lose, you know, 10% either side. But as long as we broadly get over it, we've achieved that, you know, that objective. Those two types of thinking style are very different. And I think the measurement criteria for them are quite different. Another brief example of that, because I know we're going to get on some of the metrics in a second, is let's take exposure management as an example. If you're kind of creating metrics, which in my view are somewhat navel gazing for security, which is I'm going to go take my tenant full of the scanned vulnerabilities and get to zero. Not only is that completely unrealistic, but to your point, right, I don't actually think it defines success for the objectives we should be setting ourselves in security. Instead, you know, I was like focusing on just remediating the top 20 that we did all of the exploit intelligence, probability scoring and everything that we could at the time get around instead of focusing on things like CVSS scoring and things like that. So I think that's one example where it's quite hard to almost consciously, it's not ignore things, but to it's just a form of prioritization. And I think being choice about the things you go after. I think having that understanding of that objective is ultimately what's going to underpin good measurement outcome. Yeah, 100% agreed. I think we're decent at measuring doing security things, less effective at measuring achieving security goals, to your point. So, yeah, yeah. Interesting. OK, say because I'm a massive geek and I love this area so much. And, you know, I should say for the listeners as well, Wade, you know, small celebrity being the principal architect of the Verizon DBIR, although I should say famous and infamous, maybe in the same phrase for that. So let's tackle the first one, which is the one that you and I are constantly trying to get rid of and and I think repurpose, if I say with better metrics in this space, which is the average cost per breach, particularly around data breaches. Firstly, where's the error and what should we be doing instead? And why is it still around? Yeah, yeah. All right. So so this one is this one is complicated. There's a lot of problems. So if you're not familiar, we're we're can I name names? We're talking we're talking about. Please. Yeah, traditionally, the the Ponemon IBM cost of a data breach study is where this is most famously reported. And then people pick the stats up every year. And there is a stat that is a certain dollar amount. I'm going to say two hundred and fifteen dollars per data record is the cost of a data breach. And they get this through survey. So I view that as one issue. Right. If you ask a lot of people, how much did your breach cost and how many records did you lose? I don't know that that's extremely good input to to really make decisions and measure my program on. So there's there's that aspect to. It's based on a pretty narrow range of of loss values, I don't remember what they are exactly, but it's it's it's pretty, pretty low. And then it assumes that there's a linear relationship. I don't I don't know that we want to get too mathy in in this. But the relationship between the cost of a breach and the records lost is not is not linear. It's not the same for one record or 10 records stolen as it is for 10 million records stolen. But when you do that math, you get these astronomical numbers in large breaches. And I think that's also part of the reason this won't die. In fact, if I had to guess, I'd say that's the number one reason why this metric won't die is because when you do the math and you take two hundred fifteen dollars and you multiply it by, hey, we could lose one hundred million records. That's a really large number that helps justify a lot of security spending. And at the end of the day, I think metrics are used as weapons more often than they're used to to manage the effectiveness of a program. So I think that's the real reason why that that won't die is because it produces extremely large, overinflated values that for many people are actually kind of useful. I recall a very controversial CISO dinner I was at in one of my previous roles where it wasn't actually me, which is ironic because I usually start the fights. But that there was a pretty heated debate between two CISOs. And it was over the conscious use of techniques like that to capture more budget if it was sort of duping the ex-co and the board into thinking there was more of a problem than there was. And the CISO who was doing it was arguing was, well, you know, my job is to get as much budget as possible and run my program. The other CISO was, well, our goal really is to make this organization as successful as possible. Otherwise, no one gets paid. And it was super heated. So I think you're totally right on that observation, which is that is how it's almost weaponized. Right. I think I recall when I was writing the quantitative risk framework for the Information Security Forum, this is 10 years ago now. I remember at the time sitting down with a CIO of a Fortune 100 bank and we were sort of looking at one of those studies. And it was kind of there was this weird silence between the both of us because I just sort of asked the question, have you ever seen a loss that size in the business? He was like, well, no. And I was like, he was like, have you from any other organizations you spoke to? And I was like, no, I haven't ever. And I can say 10 years later, I still haven't. And look, I think there are examples like California where you have a fixed penalty cost per record and that is a much more linear loss to manage. But I say to people the time and I've been through data breaches that when you go through the reality of one, it's typically one PDF file, one Excel file, one email that is going to incur most of your primary loss in that incident because it's affecting the one client that's going to require a lot of external counsel to manage the claim. Or it's going to be a particularly tricky file to recover or to actually ascertain what was lost in it. So a lot of forensic and instant investigation costs. It's not necessarily the fact that 10 gigabytes have been lost. It's always, at least in my experience, that needle in the haystack to your point, then it's not a linear loss to just calculate like that. Yeah, yeah. And if you if anybody's interested on Scientia.com website, if you just look, we have some recent blogs about this cost of a data breach per record. And I mean, if you just if you can imagine in your head a scatterplot with a trend line, you know what you would think you'd see as growth, just a nice linear movement. you know what you would think you'd see as growth, just a nice linear movement of of costs and records. But it's just a cloud of dots. And we did it recently and it was almost a flat line through them all, which is basically shows there's no correlation. And yeah. And and that's because of some of the reasons you're talking about there. Some not all records are created equal. Some are much more, quote, expensive than others. And it's just a bad metric for measuring the cost. A lot of a lot of other better, better approaches out there. Yeah. A hundred percent. So that's one already. Yeah. Well, yeah, exactly. So that that is. Yeah. I don't know if it seems obvious because to me, I've got to that point where my patience has snapped to the degree that I tear my hair out over this stuff, because for me, it represents quite an insular thinking of our industry, because when I speak to friends who are actuaries or friends who are accountants or in investment and things like this, and I explain to them, this is actually the differential between how enterprise risk management is conducted as a what I would call a really a compliance based practice in organizations versus risk modeling and how much of a nuanced practice that is in security. They find it really difficult to understand because then the measurement concepts on, you know, on. Well, I say this. They're not even new. Right. They've been around decades. And I know we both are obviously very familiar with and I'm sure lots of the listeners are with works of, you know, Doug Hubbard, Rich Searson and others like that who have done a lot to try and promote this through the industry. So actually, that's a good segue onto the next metric that I want to talk about, because like we mentioned earlier, I think vulnerability management is a space that seems to capture a lot of this bad measurement thinking. And the one I want to talk about or want really you to talk about, Wade, tell us again why it's poor, where we've gone wrong and what we should be replacing it with, which is meantime to remediate, because this metric seems to be everywhere. I see it in product UIs as much as I do in internal security functions in terms of reports. I think there's a lot more interest in metrics. I know Rich Searson has done a lot of work on this in his latest book, The Metrics Manifesto. He didn't even pay me to give that plug, but I just think it's a fantastic read. But yeah, tell us your thoughts on this. Yeah. So again, this is one of those. It has some truth to it. It does have some usefulness, but what people think it's telling them is not it. So I think most people look at MTTR, meantime to remediate, and they think it is a measure of how quickly the program is remediating or burning down vulnerabilities in their environment. And the faster, the better. That's the general consensus of what it tells me. So I want to drive MTTR to zero or a second if I could, because that means we're remediating vulnerabilities faster. But the thing is, the critical missing component is MTTR is only measuring the vulnerabilities that you remediate. It is not measuring all the vulnerabilities that are left open and unpatched, unfixed, unresolved, whatever. And so for that reason, you can have an extremely good MTTR by fixing 1% of the open vulnerabilities in your environment. And the measurement would look like, wow, we're blowing it out. We're awesome. We have a quick MTTR and we're secure because of that. Meanwhile, you've got 99% of the vulnerabilities that are untouched and open to exploitation. So I think it is misunderstood and misapplied for that reason. Again, it's just, let's take all the vulnerabilities we remediated and average the time it took to remediate them. A better metric, if I can add one, is to do something like survival analysis. All right, this is a longstanding statistical technique. It's been around. You can Google that and find it's often used to measure the half-life of radiation, half-life to basically anything decaying. And if you look at vulnerabilities in your environment as, I want to know how long it takes to burn those down to zero. Now, let me caveat this and say another bad metric is you've got to remediate all the vulnerabilities in your environment. Let's just say we've identified the subset that represent risk. And these are the ones that we need to do. And that's the sum total of what we need to remediate. So I want to burn that down to nothing. And I want to measure the time it takes to do that. So you can use the half-life per survival analysis to do that. And it actually tracks, all right, how long does it take to remediate 50% of those vulnerabilities, right? How long does it take to remediate 90% of them? And you can build your metrics on the burndown rate of the vulnerabilities you care about, rather than just measuring how quickly you are on those that you actually remediate. It's one is telling just pure speed. The other one is telling speed plus how close are we to the finish line, which I think is critical. Yeah. And I think when you marry that with exposure intelligence, let's call it, I think if you start getting a completely different worldview of, let's say the sort of perforated holes in your security, in your enterprise stack. And full disclosure, I began using survival analysis in the last maybe nine to 12 months of my final year in my last ESO role. I actually brought on a data scientist to help me with that. And we'll come on to a second of, do you need to be you know, a sort of Oxford or Ivy league mathematical don to be able to do this. But I mean, I understood, you know, 95% of it, but he was just far more talented at getting, getting this visually looking great in Tableau and Power BI and all the other good stuff that we were using. But one of the, I think there's loads of sort of almost sort of peripheral metrics around that space of survival analysis that I think is super interesting. Fixed speed was one of the ones that we used to really enjoy because it dawned on us as well that, so we did the same, we drew a line across essentially, you know, maybe we probably dealt with the top 10%. We did, we had a lot of exposure intelligence to calculate that, but we just knew that most of the vast majority are going to sit there and not be of any issue to us. And, you know, this was tested out over time and, you know, we were proven, you know, relatively correct. But what we then started to realize was actually, most of the vulnerabilities that we were actually trying to deal with, when again, not made equal, some required very tricky patching or workarounds, bits of control reconfiguration on our side, if the third party or the vendor wasn't able to support, sometimes their tech was coming to end of life, we had to build a workaround to it. And so the fixed speed decay was really interesting because it actually looked at the, and you mentioned this at the start, Wade, is that performance efficacy of your security program. So I began to look at, well, actually, why was my team, you know, three times slower this week? Was it because they were being bone idle, which they weren't, but, you know, or was it because actually the workaround was a lot harder and that led to then reprioritization of tasks in other areas that for me is, and I'm not saying, you know, I, we were very rudimentary in this practice, but that for me is the, what the promise was of risk quantification, which was decision engineering at scale to be able to take those kinds of informed management decisions. And to your first observation, right, I think that is one of the ways that I looked at the success of my function was the decision executions and management decisions that we were taking. Were they getting better in terms of yielding the outcomes for our objectives as a business? And so I, you know, would encourage anyone definitely listen to this. Survival analysis is certainly one of the best and most interesting ones. I wanted to, you know, briefly talk about just get your view on this because I, I bang the drum on this constantly, even today, which is, I think as an industry, we also obsess about adversarial action over event outcome writ large. And when you, and I totally get that if you're running a SOC, I can still totally understand that to a degree. I think if you're a, you know, if you're selling a cyber product in market, especially in that space, I totally get that thinking. But when you actually lead an enterprise security function, which is really an enterprise business function, you don't have that luxury. And actually, when I look back at most of the loss exposure we carried, it was accidentally generated. It was poor process architecture, user error, misconfiguration, and just sheer accidents a lot of the time. And I then, one of the first things that we did instead of just doing a typical threat landscape, we looked at an event landscape. And like I mentioned at the start of this conversation, we just started tracking all the stuff that was going on in the business. And so we began producing metrics of just like, what's our total CapEx loss? What's our total OpEx loss in the year? Like how can we increase bottom line revenue and reduce our run costs year on year on year? Do you think, or do you see or observe any security functions taking a broader view of their event landscape rather than the traditional, I'm just going to track OCGs, organized criminal groups, nation states, threat actors, hacktivists, et cetera? I do. And this has been a trend, I think. I think there was a good movement 15 maybe years ago. Sometimes I have to check my dates because I think, 10 years ago, that feels like a couple of years ago. But there was a movement where we realized, hey, we need to know more about what adversaries are doing. And we need that intelligence. And all these threat intel services and companies sprung up and adversary tracking and measuring became a big thing. And I think that's good and healthy. However, I think maybe there's been somewhat of an over-correction because you start putting it all in the adversary bucket and you forget, like you said, there's a lot that we have control over that we're doing. And it doesn't really depend on exactly who the adversary is and knowing, is it this or is it that and all of those kinds of things. So I do think that has been, and I'm seeing some positive, I don't know if pullback is the right word, but I am seeing more broader business terminology integrated into security programs. Maybe this is echo chamber going on. I'm just talking to people that are interested in that kind of thing. But I do see that a lot when I go to conferences. I have seen security leaders just using regular business terms and saying that I've been reporting these at the board level and to other executives. I've tried to kind of eliminate the security speak, if you will, for adversaries and other stuff. And performance and looking at, hey, yeah, operational expenses. What are those for our program? And how do we control that? And what's the effectiveness? So I think, yes, I think that is a long process. And we've got to find the balance point. I mean, I feel like in security, we have the problem of as soon as we start to make some progress in one area, something else pops up, like AI right now, you know, like, okay, we're at the point where I feel like maybe we're ready to turn this corner on all of this measurement, metrics, risk, not too far, not too little. We've got our Goldilocks zone. And then now everybody freaks out about AI and throw all that out the window and start over again. That's the issue I feel like we keep having. I was going to ask you about this and just throw it in as a bit of a, well, probably not a curveball, but I just wanted to get your opinion then. So one of the things that I've been doing a lot with both Perplexity, Computer, Claude, and a few others has been by creating agent scenarios in the context of this discussion, right, to kind of try and stress test our measurement thinking and, you know, query, you know, prompt the agents to think with the profile of an actuary or, you know, another business executive profile, et cetera, but to then tackle some of the scenario problems that we would still face in security and to see what it's thinking yields, thinking in inverted commas, but it's sort of its approach to, you know, it's achieving some objectives as an agent to yield different measurement outcomes. And then to try and, for me, build better ideas around, actually, are there different lenses we can look at some of these problems? Do you think maybe actually this does offer some opportunity, particularly for things like scenario analysis, where we can actually begin, I guess, to a degree, like outsourcing some of the intellectual rigor to a domain where through very basic language query, we can start asking the right questions, but getting a much more sophisticated set of answers. And I guess if you can sort of follow on to that, which is what we briefly mentioned, you don't need to be a stochastic Don or a statistician to kind of deliver this. I want you to, I know you're going to be able to some examples of how pragmatically security leaders, CISOs, whoever is listening to this, might be able to pick up some of these techniques, but just first, maybe on the agent piece of whether that actually offers a bit of a handrail for practitioners. So, yeah, I think there is a lot of promise in that kind of thing, working through thinking and solutions and possible outcomes with, in an agent assisted way. I have experimented with this on several things that we do at Scientia because, I mean, we do a lot of research and I'll just be honest, I'm not an expert in every domain that we do research. So, you know, I've started almost doing a lit review and kind of problem thinking hypotheses with with agents before just to help my own thinking and broaden that. And I think it's helpful from a metrics perspective because you're exposed to possibly different things that you didn't know about. And I think people treat agents as neutral. It's not like, oh, I'm not going to trust anything that consultant says or they're the internal person. They don't know anything, but this consultant does, knows everything. You know, there's a lot of those politics that come with recommendations, but generally people view agents as kind of neutral and maybe a little bit more open-minded to things there. And I think that's probably positive. Yeah, absolutely. And I remember certainly like 10 years ago when I was starting out, I didn't know how to sort of code and build a Monte Carlo engine and things like this, that, you know, that you can just outsource that now to agents in a way you couldn't beforehand. So for me, it's democratizing some of this discipline, which beforehand, I mean, I'm of the camp as well. I mean, I think I got, you know, certainly below average grades at maths at school and then spent my entire career obsessing about it. But you definitely don't need to be a sort of expert in that field to undertake a far more, I think, effective form of measurement practice, right? And you know, for anyone listening who's interested in this, I hope I can say, feel free to reach out to Wade and myself because we've collated quite a library, I think, of literature over the years. You know, people like Sam Savage, people like Doug Hubbard and others that have dedicated their time to thinking about measurement. We've got some super interesting examples that they've sort of pulled from. So as we sort of, you know, come to sort of the coda of our conversation, I wanted to think about some of the pragmatic things that, you know, certainly listeners hear, but I think broader audiences can take away in terms of what should people focus on when it comes to measuring things that they should fix, things that they should maybe try and achieve as objectives within their security organization. You know, what are some of the more successful approaches that you've observed, both in your and touched on in the earlier part of the conversation? Yeah. So I think this is an important question, and I would just say to everyone, regardless of what your security role is, try to think about what is it ultimately that you're trying to achieve? I mean, it should be a handful of things. If you boil them down and say, how do I know that I'm doing this well? Whether you're an analyst in a SOC, you're an incident responder, you're a vulnerability manager, you're a CISO, you know, there's a handful of things that you and your team can say, this is what we really want to do well. And then ask some questions, and don't take the surface answer, but ask some questions about how that can be measured, right? Let's take vulnerability management, since we've been talking about that for a while. We want to minimize risk exposure to the organization, I think it's got to be one of those things, right? All right, well, the top shelf answer might be, let's measure how quickly we remediate vulnerabilities. Well, okay, hold on just a second. Is that all vulnerabilities, or is that some subset of vulnerabilities? Is quickness the real thing we want to measure? So just that process of collaboratively thinking about really what are those things, rather than just accepting, I don't know, the latest metric that you heard about or read about, because I think that's important. And we have done this at Scientia. So we do a lot of research, and we do a lot of research with different types of security companies on different areas of security. And so we're in a position, a lot of times, we've got a bunch of data, and there are some things that we think we want to measure, but we're always open to what can we measure, what actually tells us something, what can help reframe the problem, and how organizations are doing it or trending toward that. And a lot of times we come up with different stuff. I'm not saying we're geniuses, but it's just the fact that we're willing to ask those questions and explore it, and there's some data there. And I think a lot of teams have that as well. They have some data. They have some questions. They have some goals. And somewhere in between there are really good things that we can measure. The second thing I'll say, and this seems super simple, but a lot of people go off track is the things that you want to measure for your job that are helpful to you might not be the same things that the level above you need to have you report to them. And then go up above that. So this, I think, throws a lot of people off. This metric is super useful to me, so it's super useful to everybody in the organization, and I'm not going to do anything else. But working through that can be frustrating because somebody else wants a different metric about what you're doing, and you're like, no, that's not useful. But remember, useful to me and useful to them, both things are useful, and you just got to figure out how to get both of them. And I think that's the way to do this in a group, in an organization, so one doesn't just get tossed out. I remember having that discussion with my former head of SOC, a very, very close colleague and friend that we'd worked together for a long time. And when I was introducing a sort of living dashboard as part of a data mesh model, so I had my data scientists work on this. This was trying to get as close to runtime decision engineering as we could get to where we could plug and play different decision sort of activities wrapped around our processes and security. And he came to me with a challenge and said, I want to measure X, Y, Z, and these things for my security operations center. And I said, that's totally fine. You can measure that. I said, but those metrics of which reflect, again, maybe efficacy improvement for you are not the sort of things that I'm going to need to take the decision at the top of that tree, right? And I think going through that thought exercise, it's hyper valuable. Another technique that I used to use a lot, which actually something that I've been trialing to bring about to what we were just talking about with agents, which is the concept of data storytelling. So taking different forms of raw data that we might encounter in security and trying to craft different narratives around them in the context of other, maybe business stakeholders, for example. And you very quickly distill the kind of, so what, like if the narrative survives and is actually interesting and can kind of meet the demand of the other stakeholder, you think, okay, like that is something that's useful to them in order to execute a decision. If it doesn't, you sort of think, right, we need to refocus on what we're measuring. So I think going through those sort of exercises, both, I think now, not only as a team is super useful in terms of a behavior practice, but I think also now trying those kinds of almost measurement scenarios, right, with agents, I think is incredibly useful just to bring to a close, Wade, maybe you could just give us a few things on what Scientia as a research institute is sort of focusing on at the moment and what you sort of see in the years ahead. Absolutely. So if you're not familiar with us, check out Scientia.com slash research, ton of reports there with lots of different security companies on lots of different subjects. Our sort of bread and butter is taking security data sets, digging into them, mining insights and sharing those as best we can. We take a lot of pride in rigorous analysis. We want to do that right. And we actually want to produce research that helps. So a lot of, we've been talking a lot about vulnerability management. So we did a lot on that over the years. I think next up, we have a report coming out with Cobalt security on 10 years of pin testing. So it's just, we get to study different aspects of the security domain and learn what we can and then share that. So in that one, we do some of the things that we've done in vulnerability management, right? How quickly does it take to resolve pin test findings and other things like that? And we do a lot of risk research. So that's the way a lot of people are interested in what we do. We're trying to measure important questions about the frequency of incidents, major incidents, how much do security incidents actually cost? And how are those things trending over time? So next up, if you want to follow us, we have this series of reports, Information Risk Insights Studies. They are sponsored by US CISA, Cybersecurity and Infrastructure Security Agency. So they're free for anyone to use and benefit from. But we're doing, we're studying the last five years of the largest cyber loss events. So the security incidents with the largest losses, the top fifth percentile. And we're doing a deep dive on what can we learn from those, right? What do they teach us? What are the commonalities? What do they say about losses and other things? So I'm excited about that one. I don't have a date for that, but if you follow us, we'll let you know. Yeah, I'm sure I'm not alone. I'm super excited to see some of those lessons learned from that research. And, you know, we'll say to everyone listening that these reports were a huge handrail when I was running security programs. And, you know, Wade, thanks so much for bringing this discipline together. I think the perspective you guys have at the Institute is pretty unique. Not only do you get to sort of correlate and analyze huge datasets across all of these really interesting problem spaces, but you're coupling it with very effective measurement techniques, which I think we're sort of desperate to be leveraging as an industry. So just finally, Wade, where can people find you, if not at Scientia.com? So, yes, I'm on LinkedIn. I post a lot on LinkedIn. So if you look up Wade Baker Scientia Institute on LinkedIn, you should find me. And I try to keep it to a bunch of data and some analysis on that. That's kind of my shtick on LinkedIn. So you get a lot to see a lot of what we do. Yeah, fantastic. Well, thanks ever so much for speaking with me today. Great conversation as always. And thank you all for listening to The Watchtower.
