Transcript
Today I'm going to talk about automating the standup of developer sandboxes utilizing Proxmox virtual environment and Ansible for configuration. At Instrumental Identity, we are an independent IAM consulting company and a SailPoint partner for several years. We've done both IdentityIQ and IdentityNow implementations, both new and existing, for our projects. One thing that we've come up with and noticed when we're working with our clients is that there is definitely a need to have a test environment. You need to be able to try out the configurations that you want to do for your customer. Or in the case of if you're working with your company, you want to be able to do your testing in a sandbox environment and not affect a production system. Second, we've found that when we're having to do repetitive tasks, it's going to take a lot of time to continually type them in, run them, sometimes you'll create a bash script. And then lastly, as a developer working on the system, we run into issues with limited resources on our machines when we're doing client work. If we have to set up several systems, those each take away from the resources that we have available to us. So, SailPoint has started providing sandbox tenants for their customers, partner tenants for partners to work with, and even now, ambassador tenants, if you're part of the Developer Ambassador program. Now that you get the sandbox, the question becomes, what do you do next? So the sandbox is really there for you to learn. It's for you to connect to services, and when you get it, the first thing you're probably going to do is sit down, load a flat file, bring in some identities, and then start configuring the system. But what if you want to do more? You could ask for some demo data to be added to your tenant, that's one way, but still you need to be able to configure your own sources, and be able to work within those systems that you need, not just those that are provided by the demo data. So this is the developer challenge. You've gained access to this great resource, and now you need to figure out how to utilize it, and you want to go further than just the demo data. So you come up with the questions, where do you install the virtual appliance? How do you get sources? How do you connect to the sources? And where do you install these non-SaaS sources so that you can work with the system and learn what you need to do for your project? Additionally, you may need to do things such as add and remove users to these source systems. And these tasks are items that you want to do regularly, but may not have an easy way to do this. So this is where I came across in my work with Proxmox and Ansible. So what is Proxmox? It's a virtual environment that allows you to spin up virtual machines, Linux containers, and also manage them by creating templates for VMs that you may want to reuse regularly. One of the benefits of it is that the community edition is open source and free to use. So it's a great resource to set up for yourself at either home or in your work environment. It's easy to learn, and the user interface is quite common, as we'll see later. It supports both Windows and Linux VMs, so you're able to stand up any type of machine that you might need to test with. And lastly, it runs on almost any machine. You can probably have seen a pile of computers similar to my image here, either sitting in your back office, on the floor in the IT department, or collecting dust in your basement. Now Ansible is really where you can start to bring in the automation portion of this demonstration. By using the standard DevOps configuration as a code tool, you're able to standardize your configurations for these demo sources and do it once and create them on multiple machines and bring in the resources that you need. If you have three MySQL databases, you can create the configuration to stand up a single MySQL database and then customize that to meet your needs. You have additional options for spinning one up and then creating a template with Proxmox. So there's many ways that these two resources can be used together. Proxmox here really becomes your data center, and Ansible becomes your provisioning and configuration engine for it. There's no need for additional tools to learn, and you can do pretty much everything with both of them together. All right, a little bit more about Proxmox. To connect it with Ansible, there's only two packages that need to be installed. It's very simple. Once you install these packages, Ansible can actually go in and create VMs directly in Proxmox from the configuration in the Ansible scripts. From the UI, you can also create VMs either from an ISO image, from a cloud init image, or you can clone a template image that you've created earlier, allowing a very quick and easy way to bring up virtual machines that you need to do your work. And the other nice thing about Proxmox is that it can be configured either as a single server or in a clustered environment with high availability. So if you have a complex system that you're trying to mimic in your test environment, it allows you to set this up using some machines that you have, like I said, just sitting around. You can put three or four of them together, and now you've set yourself up a nice clustered instance. And these also support Linux containers. So if you only need a small system for a single purpose, these are real easy to spin up within Proxmox. Now Ansible, as I mentioned, is a configuration code tool. This allows you to script out all of the configuration steps that you might need. I was turned on to Ansible when I was attempting to stand up an IAQ instance several years ago, and I wanted to combine all of the steps for configuring the base environment, setting up Tomcat, configuring a database, and adding the IAQ environment on top of that. I found that there's many tutorials and scripts that allow you to do this with just a little bit of searching. It has a standard project format, so once you get your configurations within that, it's very easy to navigate and add to that. And it allows for roles and collections. So you can utilize work that others have done through collections that they've put out in the Ansible Galaxy repositories, and you can also create your own roles that are reusable within the system to allow you to, again, configure it once and use it many places. Ansible is always expanding. There's modules being developed for all different systems, and new ones are coming up every week or month, it seems. One of those modules that was developed was the Proxmox module, and that is what makes it possible to use these two together to build this developer sandbox. And the configuration files themselves are done in YAML, which makes it highly readable and easy to configure for most users. Okay. So why do we want to do the automation for this? Some of the reasons that you would want to is just simply to speed it up. If you find yourself configuring Tomcat each time you want to set up a new development instance, then wouldn't it be great if you could just run one command and have it be brought up? You could repeatably configure your servers if you want to do updates. You can run one command that can go through and do all of the updates on your Ubuntu machines, for instance. It also helps to reduce errors when you're configuring your systems for the second or third or fourth time. You do the configuration once, you have it in the Ansible script, and when you run it again, you only need to change the configuration items that are set up for that specific machine. Now what do you want to automate? This could be a creation of a VM, this could be the setup of a whole system. You could run a cluster of IAQ instances, you could set up several IdentityNow virtual appliances and a high-availability MySQL database, for instance. You could deploy Docker with it and use Ansible to manage your Docker Compose files in addition to the other virtual machines that you're running. Additionally, if you want to bring in all of your logging, so you could write a Ansible script that will go out to all of the machines in your environment and pull the log files into a single location. And so, where do you start? You start with the common sources, the things you're going to use the most and have the most configurability. If you have a lot of manual processes in your environment, those are a great place to start and see if you can automate those using Ansible to make these processes faster and less prone to human error. If there's processes that require being run on several different servers, those are another good instance where you would want to start automating the process. Ansible, just to kind of go over some basics, Ansible has an inventory. This is where you put all of the configuration variables for your servers within your system. This is what's going to get read in when you run a playbook. The playbook is the basic definition of the script that you're going to run. It can have roles or it can have tasks directly in it, and you can have multiple playbooks to manage different tasks. And roles, these are the individual processes and the reusable bits of code that you might use within a playbook. You might pull them into another role, but the main value of these is they are reusable. And there's many roles that are available through open source means, through Ansible Galaxy, and just generally you can find online. So if you want to configure something, take a quick look online, and there's probably somebody who's already done it. All right. So let's see this in action. So we'll go to the demo. So if you guys can see my screen here, I've got the Proxmox UI set up here. So this is going to be your data center, and I've got one server connected to this right now. And under this, I have several virtual machines already configured. I have two templates that I've configured, one for the IdentityNow VA and one through Ubuntu 22.04 cloud image. And these can be used to quickly spin up additional machines as needed. So if we switch over to here. So now this is Ansible set up in VS Code. As I mentioned before, we have our inventory. And so our inventory here, we have our host file. And the host file has all of the machines configured in it. And because it's nested, you can manage them quite easy. So if I wanted to run a particular playbook and have it do all of my IDNVAs, I could easily run it with IDNVA all as the target. And that would run through all of the files. And likewise, if we scroll down, I've got the DB server set up, and I've got a Docker server set up. So how hard is it to run it? Well, let's go down here, and we want to create some VMs. So I have a playbook here to create them that says I'm going to use this template to do it. And down here, I have two virtual machines configured, MySQL DevDays and LDAP DevDays. And these are just the base virtual machines that I want to configure. And if we go down a little bit further, you can see these are all of the tasks by name that this process is going to use to spin up these machines. We don't need to get really into the details of these. That's just using basic cloud init setup to configure those. So let's go and run this. So if I grab the configuration for this. So here, you see we're calling the Ansible playbook command. We're passing it in the inventory file that I just showed you. Because we have passwords, I've used Ansible fault, which is how you encrypt the sensitive data. And we have our playbook right here. If I go ahead and run this, I enter in my vault password. And now, this is going to go through. So it looks like what happened is I have that virtual machine already configured. So if we go back to here, and if we remove that machine. So if you have a machine that you don't need anymore, you can easily remove it. There we go. So that was the misconfigured machine. Come back over here, save that, and we say, let's see here, that's the one that I wanted. Save that, and we'll go back and rerun this. And we will password again. Okay, so now what this is going through and doing is it's running through, it's set up all the configurations, and now it's gotten down to the clone VM. So this is going to go into that template that we briefly looked at before, and it is cloning this template into the machines that we've requested. So right now, it's working on virtual machine ID 510. And now that one is complete, and we can see it started on virtual machine 512. So these playbooks, each can be done separately. You can have individual items in it, it can be a single task, it can be multiple tasks. Or you can start to do more with roles. So roles would be set up off to the side. Here I have, under the roles, an IDN API folder. And so within this folder, I've started setting up requests to the API, because I can use the Ansible URI to go into the IdentityNow system, run a command very similar to how you would in Postman, and pull the data back into Ansible. And then I can use that data to further process the system. If I want to use it to maybe pipe out to another playbook, or if I want to use it to just save to a file to build a report from, I can easily do that from there. So here, I see our playbook is done. If we switch over, we now have two new virtual machines ready to get started and ready to be configured. So if we start that one, while that's starting, we'll switch back over here. So what I was talking about with the APIs is, if I go into this task, we can see that the task here first comes in, and we go get our token from IdentityNow. And then we save the access token that we receive from it. That access token then gets used further to make our second call out to our sources. So this would be the way that we can get all of the sources within the system. And if we were to run that, we can paste this in here. Again, we put in our password, and this will run fairly quick. And if we scroll up, we can see that I have Ansible going in and configuring the output here to give me just the connector, the name of the source, and the source ID. And now you can use this list to do your postman calls, if you want an easy-to-read list to get the IDs from. Or you can pipe this back into another API request, if you want to build out a big Ansible script to get all of the roles, their access profiles, and their entitlements for use later. You could build that out in the system. Now I mentioned roles before, and as you can see up here, this is not using a role. But I have created a getAccessToken role here, and under the v3, I've created a listSource role. And what's unique about this is that I'm actually calling that getAccess role directly in the listSources role to get the access token, return it, and then I can use that to get the data out of it. And so that's how you would go through and use that, and if we look at this last role that I have here, the getAllValues, this one is going to go through and it calls directly each of the three roles that I've created up here, and it will get me all of the sources and print it out to the terminal window. So again, I can then take that, use it with another system, or save it for reference for myself. Really, you can build out whatever you need, and you can also create a template for your body for JSON if you want to take some of the data that you're getting from your source system and push it back into SailPoint in IdentityNow. So if you have a MySQL database that you configure and set up, and you have a template for the source body that you would send in to create that source, you could have Ansible, after it creates the database, push that source up into IdentityNow, and you'll be ready to go for your next test environment with very minimal external work for yourself. So that's briefly about how you can use Proxmox and Ansible to do your configuration and stand up of a sandbox system for sources. Hopefully you found that interesting, and I will expand on this more in the blogs in the future.
