Transcript
Mike Matchett: Hi Mike Matchett with Small World Big Data and I'm here today to talk with Constellation Network about some of the cool stuff they're doing in AI. They have a cybersecurity background. They've done a lot with blockchain. We'll get into that a little bit. But what they're focusing on right now is how do you secure AI itself, your usage of AI? How do you keep things like PII from going back and forth? How do you create an auditable log of what's going on? And even how can you clean up your token stream to make you some money on this and recoup some efficiency? So hold on a second, we'll get right into it. Hey, Alex. Uh, welcome to our show. Alex Brandes, CTO, Constellation Network: Hello. Yeah. Excited to be here. Mike Matchett: Uh, so, uh, this is Alex. Alex Brandis, and you are with Constellation Network. You're the CTO over there. Uh, you guys have been working on some complex problems, mathematically complex problems in the less. Uh, for about nine years and you've had this company, but you start out kind of in this blockchain space. What drew you to doing things at that end of the spectrum and hard problems like blockchain? Alex Brandes, CTO, Constellation Network: Yeah. So we, we launched about nine years ago, constellation network. And it's a blockchain network focused on, on data. So most, you know, existing blockchain solutions were all about currency. So just cryptocurrency, but we, you know, we saw currency as kind of a special case of a bigger problem, which is data. So we, you know, we built a, um, a network from scratch. It's a custom architecture to focus on data processing, um, bigger data workflows. And, uh, um, yeah, it's been, it's been an extremely challenging, but it's been, it's been a ton of fun. And, uh, here we're, you know, using that background and, uh, doing some products in different spaces, but specifically in AI these days. Mike Matchett: All right, let's get into that. So you kind of look at and say, you know, blockchain as most people have a mental image is with cryptocurrency and that. But when you just as you said, as you're building out your own blockchain solutions for other things, you solve a lot of the other problems that become fundamental when you look at now, what's to say, for example, going on with AI, right? You have this idea of building a resilient network, a scalable one, a bulletproof one, one that's completely auditable. Uh, and you have now sort of this experience corporation wise to do this. So when AI starts to emerge. Tell us, tell us this part of the story when AI starts to emerge, what did you look at there and say, hey, this is where we can make a difference. This is where we should dive in with what we bring to the world. Alex Brandes, CTO, Constellation Network: Yeah. I mean, I think we, uh, or at least I specifically, uh, had the experience that a lot of, uh, people had that were paying attention to AI around the end of last year. Is that at a certain point, maybe December 2025, the models were just significantly better. And all of a sudden, um, you're able to really rely on those models to do a lot of heavy lifting and a lot of a lot of work. Uh, and so, you know, all of a sudden the software industry is just completely turned over on its head. Uh, no one's writing code directly anymore or they're not writing the majority of it. Um, you know, AI agents are writing that, that code for them. And at the same time, there's all sorts of, um, just general, uh, you know, non-coders knowledge workers of, of various types that are kind of branching out a little bit farther than they're comfortable with, with tools like clockwork or similar ones from ChatGPT. Um, and they're kind of stretching their technical ability to do more than they could before because of these tools. Um, and what we saw is that there's this big gap in terms of security, uh, people are doing more than they were before, but they're unsure of what the security implications of that are. Are they sharing, um, you know, personal data that they don't want out there? They're experimenting with frameworks like open core that are autonomous agents. Uh, and, you know, they're giving them access to email to all the, you know, all this other data where there's, uh, you know, the possibility of, um, prompt injection attacks, which would be, uh, attacks where, you know, some data that the, the model ingests tries to trick it into revealing private information or behaving in a way that you, you don't want it to. Um, and so we saw, you know, we saw an opportunity to, um, enter that space, uh, and specifically create tools around AI security. Mike Matchett: All right. So, uh, clearly the wild west of AI, everyone out of coding and doing all sorts of other things with their petabytes of corporate data and giving their agents access to their APIs and not knowing what those agents are doing needs, parental control, really needs someone to to sort of lord over what's going on and look at the communication between and step in occasionally and say, you know, no, no, you can't, you shouldn't do that. And to do that. So how do you how do you how do you then impose those controls? What is gait AI look like? Uh, which is, you know, what, what, what you've just sort of released here, uh, in practice. What is it? Is it like a firewall for AI? Is it a gateway? Is how should people think of this? And what is it? What does it look like if I, if I use it? Alex Brandes, CTO, Constellation Network: Yeah. So we, we call it a, a drop in security solution. So it's, it's, uh, essentially as a proxy between your agent and the model providers that, uh, it depends on. And so because it's in, you know, gate is in this, um, position on both sides of the request, uh, we're able to inspect each request. Uh, we can redact, uh, PII data or API credentials. Um, and we're able to do prompt injection detection. So detect if someone is trying to trick your model and doing something you don't want it to, and can actually block those requests. So they never hit the the model. And you know, then you can prevent any, any possibility of those, those attacks affecting your, your agent. Mike Matchett: So, so for example, if I'm, if I'm a fast food restaurant and I don't want people using my order chatbot to do their Python homework, you can look at that and detect, for example, Python questions and weed those out. They should just be ordering hamburgers, that type of thing. Alex Brandes, CTO, Constellation Network: Exactly. Yeah. So, um, you know, we're able to sit in between essentially support any local agent. So it could be cloud co-work, cloud code, um, codex open code, all sorts of different frameworks. Uh, we just, you just route your requests directly through gate. Um, and then we can prevent things like, uh, having the Chipotle chat do your, your Python homework. Mike Matchett: So this seems like a, so a sort of a centralized security point in the infrastructure where, say, if I'm an I T, which is a lot of our audience is in in the IT side of the house, they can get their hands around the traffic that's going between the users and the AI service providers and basically stop data exfiltration or data data leaks, especially bad data coming in back to their users to keep them from spreading it around or using it. Um, and, and how does that is that, is that based on, in turn, more AI use or machine learning or patterns? What's just give us a little brief overview of what are the guts of that? Alex Brandes, CTO, Constellation Network: Sure. So, um, you know, for the prompt injection solution specifically, we, we recently released, um, some, some benchmarks kind of showing how, showing that it works and proving that it works. Uh, and, um, you know, beating a lot of the kind of enterprise players in the, in the space or matching them at a, at a lower cost. Uh, the, the way it works is it's a multi-layered detection system. And so we don't, uh, don't reveal all of the internals exactly how it works. But, um, there's, there's different models of different types being used, uh, to kind of inform other layers, uh, and then be able to, um, to do that detection. Mike Matchett: Okay. So kind of, it's pretty sophisticated, much along the lines of what you were learning to do with the blockchain stuff that we talked about earlier, where you start, so you get some of this heritage expertise being applied here. Uh, I one of the things that I, I'm a long term capacity planner. I used to do capacity planning for some big data centers back for some other companies. And one of the things we were always trying to save the money, right? It's like, how do you, how do you use your resources to their best effect? And one of the problems we're seeing with AI, of course, is token tokens are starting to cost real money these days. And the prices, prices going up, uh, we all expect them to continue to go up. So to optimize token usage is a big thing. And you can do that too. I understand there's, some part about sitting in the middle of this token stream that allows you to help people become more efficient, right? Alex Brandes, CTO, Constellation Network: Yeah. The primary way that we, um, that we can make an impact there is through prompt compression. So, you know, there's, there's lots of, um, things that are sent in a request that don't necessarily have any impact on, on the model, but they do cost tokens. Uh, and so because we're in this position where we're sitting in between you and the model provider, uh, we're able to, um, add compression, remove some of the things, things like the most basic example would be like white space. Um, but there's a whole lot of specific formats that, um, that we support that can be removed. And then we're able to save on average about 20% of token usage if with, with that solution. So the, you know, the, the cool thing is that, you know, you, you just, uh, you set up gate, you point your agent to it and you get this compression for free with no, no impact to the output of the model. Mike Matchett: Right. So I'm getting security, I'm getting auditability. I'm getting some efficiency gains out of this. I'm getting some actual revenue recognition back by spending less tokens, which is cool. Uh, it's also sitting at that, that, that core center center, like between all the prompts coming in, um, and the, the service providers with the models, you can arbitrate which model you're using on the back side to a little bit. Right? Or is that, is that something people are asking for? Or how, how are you approaching that? Alex Brandes, CTO, Constellation Network: Uh, yeah. So that, that is a direction that we're, we're moving and developing a solution for, for auto routing. Um, so there's, there's two ways that you can use gate one would be as a, bring your own keys customer, so you can keep your cloud code subscription or your ChatGPT subscription and just route through gate. Um, the other is, uh, the other way to use it is as a direct gateway. And so we would be your, your model provider. You could get, you know, hundreds of models, uh, through that gateway. And as part of that, uh, there's an auto routing solution that is going to be able to choose the most efficient model to serve, uh, each request. And so, you know, we see a lot of folks using, you know, the top most expensive models for everything. Maybe it's opus or, or fable. Now it's even more expensive, um, doing things like, you know, summarizing a long piece of text when you could do that with, uh, something like haiku if you're in the cloud, uh, world or, um, there's a lot of really capable open source models these days that can do it at something like 100th of the cost. So we expect to see usage of open source models increasing as, as time goes on, especially as the, um, you know, state of the art models become more and more expensive. Mike Matchett: All right. So another way for a company to save money today, but also look downstream and say, hey, as our users start to use more of a pantheon of models, as it were, you can intelligently route their requests and they don't have to have the mental images of which of the hundred models I should be using on each request. They can you can sort of offload that task, which sounds pretty cool. Like, and I know we're talking about cost, but you know, one of the big things for some players today is just quota. You know, they're just they're running out of tokens. So if you can save 20% on their token usage, you know, that's another day in the week that they can keep working, uh, on that. So that's kind of cool too. Um, so what, uh, what, uh, what comes next? Uh, you mentioned open claw. How, how are people using networks of agents and does, does gate AI still work? When I farmed out my work to like a lot of different agents, do they all do. They all interface with gate? How does that work? Alex Brandes, CTO, Constellation Network: Yeah. So that's one of the, you know, the, the major use cases for gate is, you know, if you're running lots of different agents in different places and maybe you're using different model providers on the back end. So you've got a ChatGPT subscription, you've got a cloud code and maybe open router, and you can route all those through gate as a central control plane. And so you have observability into what all those agents are doing in a single place along with, you know, any security instances or, um, you know, personal information that they may have attempted to share. And you can, you can track all that down through, through a single place. Mike Matchett: So I can't, I can't tell who's gonna be more excited by this in, in the IT space, the people doing, trying to get a security handle on AI and the agents or the people trying to get control of the money and the usage of the AI and their token costs, or the end users who are suddenly able to use really proactively better use 100 different models without having to know what each one of them specializes in. Alex Brandes, CTO, Constellation Network: Yeah. And that's one of the things that you can do through, uh, through the platform also is track spend and put controls on on quota for your agents or for your teams agents as well and say, hey, you've, you know, you've used more than you should this this week or this month. And so you're cut off. Or you could also just flag it and say, okay, we should, we should take a look at that and see if we can reduce it. Mike Matchett: In some companies are like, use more, use more. You're not a quality developer unless you're maxing out your whatever. We'll see how that goes. As a capacity planner, I'm like, yeah, that's not the right. Anyway, uh, if someone wants to learn more about gate AI, uh, and what you're doing over there at, uh, constellation Network, uh, and maybe, uh, dig a little bit deeper into saying how they can use this. And I assume it's an easy to adopt and implement solution being a SaaS service, but what would you have them start, start the research? Um, what is there any particular learning path they should follow? Alex Brandes, CTO, Constellation Network: Yeah, they can go to constellation gate.ai, uh, and read more about the product. Sign up. There's a free tier. So if you want to just, um, you know, try it out and see how you like it without, uh, without incurring any cost. You can, you can do that as well. Mike Matchett: All right. You guys got it. And, uh, I'll give you, I'll give you sort of one last thing here. Uh, Alex, if, if you had to give one piece of advice to someone who's deploying AI today within their company, who is in charge of securing it, what would what would you tell them? They should really just focus on what would be your sort of best practice recommendation? Alex Brandes, CTO, Constellation Network: I think the main thing is just, um, having visibility into what's, what's going on, you know, how are, uh, your agents interacting out in the world? Uh, how are your employees, agents interacting and using, you know, uh, private information or company information? Um, so just having that observability and then, uh, you know, security is, is, is a huge deal. Uh, it's, it's an evolving space. And so there's going to be constantly new forms of attacks and new new ways that, um, your, your private data could be leaked through these agents. And so, you know, there's no reason not to have security solution like gate in between, um, you know, your, your private data and the rest of the world. Mike Matchett: Okay. Thank you so much, Alex, for explaining that to us. This is a really nice, uh, dive into, you know, sort of state of the art of what's going on in how you use AI and use AI effectively and efficiently and securely. It turns out. So appreciate you coming by today. Alex Brandes, CTO, Constellation Network: Yeah. Thanks so much for having me. Mike Matchett: All right, check out constellation Network Gate AI because it can really save you money and save your tail perhaps, and certainly save your company some secrets and save a bunch of things. So check it out. Thank you so much for watching.
