Transcript
So we're going to be having, we're going to be having a talk on setting up Secure Secrets Management using Vault, Packer, Terraform, GCP, and Kubernetes. This is like a lot that I put together, but we'll just go with that. So the main thing we'll be talking about here is Vault, right, because that's like the main point. So every other thing is just going to support our process to this. So a quick introduction, I am Celestina, I'm a cloud engineer with MoneyPoints, like one of the biggest fintechs in Nigeria, or in fact Africa in short. So I am also a co-organizer of Ashicorp user group, so those are my details if you want to connect after the talk. And we'll be dividing this session in a few steps. So that's what Ashicorp is, Ashicorp Vault is, how it works, why we need it, automated secret engine, what tools are used, then code walkthrough, and then a very quick demo. Yeah, a very quick demo. So what is Ashicorp Vault and how does it work? So I picked this up from Ashicorp documentation online. So it's just a tool that organizations use to manage secrets, sensitive data, and store and encrypt secrets. So secrets are stuff you do not want to share with anybody or whatnot. You want it to be secure, and they are important for your apps or for your services. And Vault provides a way where we can secure the secrets and make sure that they are not exposed to just anybody that can access it, right, with the right credentials, authorization, et cetera, et cetera, that they are going to get in. Okay. So how does Vault work? Vault authenticates and authorize access to secrets based on identity. So this means that if the service or the app that is trying to, or the user in this case is trying to access Vault, if they do not have the needed authorization to access the secrets, it's not going to produce the secrets for them. And then also we could use Vault to encrypt data in transit and at rest. So Vault also has an audit log that we use that is used to audit, assess the way secrets are being moved out, who accesses it and whatnot, and also prevent credential exposure, eliminate and prevent data breaches, consolidate credentials and whatnot. So an easy for the DevOps guys, because they can just integrate this into their code, into their CICD pipeline and not necessarily have to write or put the secret itself on code that you eventually start to GitHub, which is risky or is not good enough. So why do we need Vault? So I may have mentioned something about that while starting. So instead of storing secret directly on configuration files, we use Vault to centralize them and reduce risk of exposure. Reduce risk of exposure, then access, there's like granular access. You can give access based on what you want it to do. Do you want the user or the app to be able to just list the secrets? Do you want them to reach the secret? Do you want them to be able to update the secret? So secret rotation also, then for dynamic secrets. I like this dynamic secret most times when I'm working with databases, like you make sure that even the DBAs don't have access to those credentials that they think they have access to, then audit logs and whatnot. So then we'll be talking about the authentication method. We'll be using Kubernetes because that's what we'll be connecting with for this. And then the secret engine for this, we're just going to be using a basic key value pair, which is this here. So what tools are used? So in the title, we saw we had Vault, we had Packer, we had Terraform, we had GCP, we had Kubernetes. So for this, the Vault would be what would be storing and managing, controlling the access of secret, while Packer would help us create this image that we can use in our Terraform file so that we don't have to always go into the Terraform, go into the VMs instead, and then start installing Vault, setting it up one after the other. So Packer packages this, and they make sure that it is in an image format where we can now pull it using our Terraform. And GCP would be the cloud provider we'll be using for this, and Kubernetes would be where our services would be hosted. Unfortunately, I don't know why I cannot connect to my Kubernetes cluster right now, but I'll just go on with this talk until I get there. So code walkthrough. Then after code walkthrough, we do a light demo. So for the code walkthrough, I'm hoping everybody can see my screen. I am using Git actions just for simplicity's sake, and I'm using two flows, which one would be to view the image, and then one would be to deploy the image when the image is viewed. And then there is this one, this would be the service account that our cluster would be using to connect. So on the cluster, we would be creating a service account and a cluster binding. The role it is going to be binding with would be created on Vault server itself, while this would be created on the cluster. So let me just close this thing. So this deployment is a sample deployment I created for us to just use for test purposes. And this policy is just like a simple ... Okay, so on this policy is a simple policy just for read for the Vault parts we wanted to read, right? And then here, the role, you would apply this on your CLI. I just prepared this so that anyone that is reading this later or watching this after can go through the GitHub repo I'll be dropping after, and then use it to deploy when you want to. And our Terraform file. So in this Terraform file, I've defined the buckets in which it's going to be in. I've defined the variables I need, the providers I need, the image. So this image will be pulled from the image that is created using Packer from the project. And I created a network in which it would be, the sub-network, the firewall, the compute instance. This is just like a basic Terraform file just so we can use to test our instance or the image that was created using Vault, right? And then here, the secret. So here is the secret, sorry, the install script that we'll be using on Vault, on Packer, sorry, on Packer to use to install the necessary things we need to install. So for this, we'll be using the Vault of version 1.8. It will update it here. Then it creates a user without access to home for extra security. Then it adds necessary permissions, creates necessary files, do export environmental variables. Then for this also, we'll be using the storage, we'll be using file. So file will be the storage we're using. So this file, this part, it was created here, right? And then we're using file for it. So this KMS key has been created on GCP, and then the keyring has been created on GCP before now. So it will be connecting to this, and then this is a simple service file that it would use to start up when the server is up and running, then a simple reloading and whatnot. So this is just like a repetition. So then there is an Helm file. So what this Helm file would do for us now is it picks up this chart, which is Azure Cloud Vault chart, and installs it into and create a release for us into a cluster, right? And this other chart is going to create the service account and cluster rebinding we need, which is charts, charts, charts, yeah, charts, yes. So it's going to create everything that we have here, right? So then, yes, then the Helm file. So now this is a, let me come here. So I used GitHub Action for deployment, and then when this is done, it moves it to GCP, and then both for creating the image and deploying it. So on GCP, this image was created. So a simple rerun, I didn't want to like start pushing this here, but the code works kind of like out of time. So the code works, and so if you have this code in your repo, you can clone the code, have it in your repo, push it, it'll work. So the only thing you need to change, let me show us briefly here, okay, yeah, yeah, here is your Google credential. So you need to update this on your GitHub page. So you update the secrets with your service account key, so that it will be able to connect to the GCP console, okay, okay. So then we come here. So this image was already created using that file, it was created, I think, this morning or so. It was created using that file, and then it was also a rerun of this, a rerun of deploy also created this. So then we have this SolrAuthAccess, initialize this, okay, okay, so I think this one was created newly, as I was trying to do some adjustments. So we'll do that together, we'll initialize it, and then we'll do this, so I can see it. It's taking a while, this is why I wanted to have everything set up before this. So with this, there's votes on this already, so vote, okay, so we have our, don't worry, this is not, this is for a demo, so it's not something to really stress about. Everything is going down after this. So I said, we'll be using Kubernetes, we'll be using KV, Secret Engine, then the access, the auth method will be Kubernetes, so to authenticate, we could do it from the CLLibre or the front of it, let's just do from this, the name would not have to be the name, then we can set up a simple secret, let's just do, vote. So our secret is created, and so, I think I'll have to adjust that one, because that's not what is in my deployment file, okay, so the auth method would create the one we need, which is for Kubernetes, then we'll enable it, so to configure this, I did all these configurations before now, but here we are. To configure this, we'll be needing the clusters, I think this is for the cluster, okay, we'll be needing the cluster's IP and certificate to be able to reach it. Okay, so now we've enabled an authentication method in which vote server would be used to connect to the Kubernetes cluster that we've set up before now, and then, what we now need to do is, we need to come to, we need a role, so yes, so this is where this thing is coming, right, because it just makes it easy for you not to, where's my role, okay, okay, so we'll do this, oh, sorry, that's gone, where is that, so this writes a role for us, yeah, it writes, it creates a role for us, if we'll come here, we're going to see that a role has been added to this, and in this role, we are using a policy, the policy is not yet created yet, but to do that, we just need to do this, oh, yeah, so you see, the KV, I did, that should be KV, because I didn't change that, so what I mentioned then, okay, so now we have a policy that the cluster, yeah, policy, so now we have a policy that the role will be using to access it, so from here, there's, let me see if I can access my, for some weird reason, this has been spinning for like five minutes before I joined the call, and I really don't have enough time to troubleshoot it before this, so what I would have done would have been to take this, this service account, apply it, so when you apply this HEM file, a simple HEM file sync would sync all this, it would create the service account for us inside this, we only have one service account here, it would sync it and then create a release also that would be the bot agent injector that would be used to inject secrets into the services that need them, then, yeah, okay, then after that, there'll be like an annotation, yeah, I think I have, I have the code, sadly, I can't do a demo for this right now, so, okay, I think that will be all, because I really lost access to the cluster before now, so I would be dropping like a full read, read me on my GitHub page, so you can use that and follow through with the complete setup and you'll be able to have like an authentication between your cluster and your VPC, thank you.
