Transcript
to build their initial infrastructure. You provision servers, you provision databases, everything you codify, Terraform obviously is a solid foundation to build on. So we're going to dive a little deeper into this, make sure we can show you some ways to enhance your workflows. We'll have sweet animations going on, but there's also a lot of learning, so I really hope the coffee is kicking in for you. We'll build up our workflow in three different phases. The build or day zero phase, you'll hear those terms a lot. You heard them during the keynote. You'll hear them during my part, also during van session. Build phase, you're defining your infrastructure as code, collaborate with your team on reusable modules, and keep building on that. Keep iterating. Keep making things better. It takes you to day one. This is where things get really fun. Take these modules and use them to standardize deployments across your organization. This allows you to build reliability in, you enable best practices that your whole organization can use, consume, and build on, keeps the CISOs happy, keeps the CTO happy, shareholders as well. This is all great. You might even start enabling self-service approaches. This is where things get interesting for other teams, because when you drastically decrease the wait time for an application development team, developers get happier. I mean, that's the service of the platform team, right? But all of this really sets you up for success as you grow beyond your day one, day two operations, and the foundations you've built during those first two days are really what help you unlock everything that you need as you scale. Get better insights into your infrastructure, get better scaling, understand the health of how your system at large is operating. And of course, reduce time, cloud waste, reduce spend, or make spend more intelligent. So you're not just deploying infrastructure, you're deploying the infrastructure you need. None of this will happen overnight. We call them day zero, day one, day two. This is not a Monday, Tuesday, Wednesday thing. This is sometimes a multi-year thing. That's totally okay. It's a journey. And through our experience with thousands of customers, we've seen this three-stage blueprint emerge. We're really stage one, adoption. So individual teams solving individual team problems. And often with ad hoc approaches, stage two, we already see the concept of a platform team emerge. This is exciting. My code now gets used by more teams. I mean, you know this, if you've ever written a Terraform module and more people start using it, you start to put more work into it. Yay, my code gets used. I guess nobody here has that feeling because you all don't have Terraform modules. Nobody here builds modules? Quick show of hands. Come on. The coffee has kicked in. I know it. There we go. Thank you. But you're not a platform team yet. You're not fully formed. This is not your ultimate form, and we're getting there though. Stage three, when you're scaling, you bring more resources in, human resources, infrastructure resources, everything, and your self-service patterns and workflows are fully unlocked. You can focus on optimization and everything that goes with that. So in my session, what I'm going to focus on for here is really just the infrastructure blueprint. But this is only part one of a story. Part two is where Van is going to talk about the security part, and to get the whole story, you really need to see both sessions. So don't dip out. For my session, I want to focus on two personas. On one side, we have the application development teams, want to make sure their app gets out as quickly as possible. It might be a mobile app, it might be a desktop app, it might be any kind of service that needs to go out. And of course, on the other side, we have platform teams, want to make sure anything that goes out meets all the criteria we have. Are our modules correctly built? Are we using the right compute instances, the right sizing? Are we making sure our keys are properly rotated? Are we actually using encryption? These are all things that matter, but oftentimes as an application development team, that's hard to figure out because you might not be trained in that. You might have the intent of building the best possible application, but we need a little hand-holding and that's okay. None of us are experts at everything, we cannot be, but together we can focus this out. It really comes down to one thing. This is not a competing workflow. Left is not left versus right. It's not app dev versus platform. It is app dev working and being empowered by platform. Bring peoples, sorry, well peoples, of course, because we're deploying multi-cloud, multi-region, multi-continent, but also bring people and processes together, use the right tools and you can unlock this workflow. That really brings us to our first question. How do we go from this desire to not be a blocker to becoming an enabler? And all of that really starts on day zero, the build phase. Let's start with a high-level look. That's exactly what I need, on how we can combine HCP Packer, HCP Terraform, and Waypoint to build the first few things. We're going to keep this a little interactive because I know heat and coffee is hard. Who here is using Packer? Keep your hand up. Beautiful. Now, who of you has used Packer to build Docker images or any type of container images? Saw somebody there cock his head, like what Packer can do that? Packer can definitely do that. You should absolutely do it because it's beautiful. And that's what we're going to do for this session today. So we had Packer build our Docker image. We publish it to the HCP Packer artifact registry, and now in Terraform, we're consuming that image. We publish a module in the private registry in HCP Terraform so other teams in our organization can use it. And finally, we enable no-code provisioning and make sure Waypoint can consume the module. I know we all love animations and diagrams, but this is where I feel at home. We have a nice editor interface. So we'll start out by defining a data source. And in our case, we've got the HCP Packer artifact with a Node.js base image. Our platform team has already ensured that everything here is correct. We've got an allowed version in there. I think it was Node 20, which is still barely within acceptable limits, and we're building. And you'll notice that I'm starting from this trusted source. That's important. I, as the application developer, can build on what my other teams have already built for me, and I can consume that. And then as we continue on, what you'll see in a second is the top part really is a relation. This is a parent to the child image that I'm going to be building. So as we switch to the actual Packer part of defining a Docker image that we want to build and ultimately publish, we can use that relation, get the image digest, build on top of that. And of course, as part of this build process, we're going to be adding a couple more things in. We're going to define an owner, a department, things that are important, tagging, you know things that are important when we want to figure out who's actually creating costs in our infrastructure. If you're scaling up to a couple thousand, a couple million resources, not knowing who runs what and who built what can be a very bad experience for your team. This is not the kind of research project that you want to be put on because it's never fun. And of course, it's a modern web app, so I need a shell script to build all my webpack stuff and download everything on npm.js and make sure the image is at least 14 gigs. So eventually the image gets built and the beauty of a Docker image is that we only need to have it in one place with AMIs and other things. We actually need to copy them cross-region. But here, single image, v3 was the previous one. We're going to select v4 for development because that is the latest and greatest. And make sure the development channel is correctly updated. Now over to Terraform because UIs are important, but really it's infrastructure as code, not infrastructure as UI. So you can see that I've already defined an ECS task, Elastic Compute Service, Container Service, sorry, still Compute, but definitely container-based, where we're referring to that very same image that we created inside HCP Packer. We already have Packer creating artifacts that we can use inside Terraform. And because if you've used Terraform before, you know this as well, because this reference is dynamic, I don't have to go in there and change this. In the previous slide, we selected what's the right channel. This grabs the correct artifact based on that. And of course, as we build more and more and more modules, this kind of reusability becomes critical really quickly. Because how we standardize, and not just standardize, but optimize our modules, defines how we reduce risk. That is not something you can ever do ad hoc. Ad hoc changes are usually riddled with good intentions. And that's what the next outage after action report will call out and say, well, this went wrong because we had the best intentions. Doesn't work that way. We had Stefan on stage talking about how aviation approaches IT. As IT operators, as reliability engineers, we would do well to learn from aviation. Because if there's one system that cannot fail, it's probably in that industry. So best way for us to deal with this is bake compliance and security best practices right into your code from day one. And of course, we try to shirk this responsibility for the past 10 years, but we cannot anymore. It's part of best practices. We as an industry shirk the responsibility of testing our code. But we cannot do this any longer. If you want to have a best practices module, best practices code base, you need to have checks and tests and assertions in your code. Application developers write their integration tests, write their unit tests, Terraform tests, and the Terraform testing framework now gives you those tools as well. So start building them. It is worth it, and it is important. It's a huge milestone once you've started adding tests on the path towards producing more high-quality software. Yes, your module is a bunch of text files in a specific language, HCL, but that code is very much software, and usually the software that runs your company. With the Terraform test framework, you can ensure that every new module version gets automatically tested with all the things you need. God bless you. These checks are really a great way to get your day-two operations much more solid, much more reliable, and easier to understand. In this case, we're focusing on a health check. We're making sure that our service that comes up for the ECS task that we defined earlier returns a 200. And that really tells us that the container did not only start, but it is actually serving something that we intend to serve. When you think about it, Hacker builds the image. The Docker plugin reports to Packer, I've successfully built this image. That does not necessarily mean your web server inside the image is starting up correctly. You might have changed an entry point, you might have changed a command or CMD instruction. Only when you get to this part of having deployed the module can you actually reliably say, this deployment was successful. And once you've got that, you have your code complete, you can publish it to the Terraform private registry, HTTP Terraform Cloud private registry, and make it available for the rest of your organization to consume. Now, here's where it gets fun. Writing tests is hard. Writing tests usually starts with, I'm going to write a few tests that I know that are a happy path, because that's what I really care about. We understand that, we've dealt with that inside the private registry, we've got a function that uses AI. It's not just AI smeared on, it's actually useful, that allows you to generate tests. And you'll get a handful of different files that you can download, single click, add to your repository, and later on, HTTP Terraform will run those tests for you. And of course, the other important thing, we want more people to consume our module. Everyone in here is obviously a Terraform certified module author and operator, but I know we support people who don't understand those things, who don't necessarily want to learn those things. And this is hard, right? You can have the best module that deploys the perfectly orchestrated EC2 instance that you've been dreaming of since 2003. If nobody else understands why you need to set 178 variables, that module is not going to get any love. Not speaking from experience here, I usually top out at 100 variables, but you see some weird stuff happening. So remember the tests and checks we wrote before? Continuous validation inside HTTP Terraform gives us those in a nice and visible way. We'll get into that in a second. But first, let's switch over to HTTP Waypoint. Waypoint templates are really what unlock all of this for us. We've got our no-code Terraform modules, so I can write what I'm comfortable with. I enable a few flags in the UI or in the API, whatever way is most comfortable for you, and can say, these are the defaults that I want to set up for my team. No more 178, no more 100 variables. I'm exposing, I think at this point, like four or five. Much easier to consume. Because realistically, we know all of those variables. Most of them we just set for us, because we know hard-coded references are bad. But not every user does know that. So Waypoint template, still ultimately orchestrated by HTTP Terraform. Users get a nice, app dev users get a nice UI. We get the comfort of HTTP Terraform. Everyone wins. That is a lot of features to talk through. Use Packer to build images, publish them to our HTTP Packer registry. Then we authored testable, this is important, testable Terraform modules that we publish to our private registry. And of course, enabled no-code consumption through HTTP Waypoint. And even in this early phase, we're already seeing a couple of benefits. We're still on day zero, but we're able to lower costs, lower risk, and increase speed. These are great things when you're doing your self-reflection, telling your manager, hey, these are the things I did for my company. Take note of this. You know, evaluations are coming up eventually this year. You really need to ask yourself, how do we make day one and day two even better? As we move into the deploy phase of our workflow, this is where it gets really, really interesting. HTTP Terraform brings you consistent provisioning. We know this. We have around 4,200 providers at this point, 4,225 yesterday morning. I don't know what the number is today. That number keeps going up every day and it's beautiful. Because it unlocks a single tool for every possible workflow, sorry, for every possible deployment that you have in your mind. And for us today, we're going to use one way to consume that, which is HTTP Waypoint. Of course, other teams in your organization might use other service catalogs, ServiceNow, Kubernetes Operator, maybe even traditional approaches. But no matter what the left side looks like, the center all converges around HTTP Terraform. It's your consistent execution environment. Policy enforcement, optional validation of deployments through run tasks, pre-apply, post-apply, lots and lots of things that you can enable to make the reliability of your infrastructure go up. And so as we switch over to HTTP Waypoint, we get this curated workflow because of those Terraform modules and because of the reliability we built in that application developers really, really enjoy. I'm not an application developer and I enjoy being able to just click the few things that I need because I know all the silly variables that I don't need to ever set are already set correctly. Behind the scenes, HTTP Terraform, orchestrating everything, bringing together the right exchange of inputs and outputs and making sure it works. But as a developer, you see none of that. The reality is if you ever helped a service team, there's always a few little things they need. Reliability is the kind of thing that was not written in the spec. Oh, well, I just assumed that my web application, which only says in the checklist, needs HTTP serving, also needs a Postgres database that's HA. Okay, well, it's an annoying change and of course we have to refill in 36 forms, but we can also just click one button, enable that same workflow through a module. And of course, these add-ons, just like templates, are powered through Terraform modules. This is not a new tool in the sense that, oh, you need to rebuild everything that you've built. This is an enhancement to everything you've built that you now can use in a different way with a different audience and make them be as successful as you. If they win, you win. I like winning. I don't know about you. So in the deploy phase, application developers get to use self-service to create and consume the infrastructure they need, all through HTTP Waypoint. And just as before, we can see and reap a lot more benefits. From a cost perspective, higher speed means less cost, usually. But because we have the reliability built in, it's not just higher speed. It's higher reliable speed. You know, going 250 on the Autobahn is great. If you have a car that you do not trust, that's not going to be a great experience. Speed at the cost of risk is not worth it, because it's not speed, it's just a bad experience. Speed with risk properly mitigated is great, because it also keeps the cost down. If your car doesn't crash, if your infrastructure doesn't deploy incorrectly, you're winning. And that really brings us to our final question. What happens after day one? If you've been following along, you know the answer is day two. That's correct. I would not fault you for that answer. But we really want to figure out how do we scale from this and go further, go bigger. You may call it day two operations. You may call it scaling operations. You may call it, this is like the life phase where our service is no longer nascent, no longer a child, but it is grown up and it's now running in production and everything. Irrespective of the term you use, this is hard. Scaling is hard, because there's a lot of little actions that come in that in a self-service platform cannot be something where I can just open a Slack message a hundred times a day and ask the platform team to do things for me. So one way we want to solve this for you is giving you tools that you can then expose to your developers. HCP Waypoint Actions is one of those tools. It allows you to recreate runbooks that you run, such as triggering CICD workflows through GitHub Actions and webhooks, kicking off Terraform agents, promoting, and indeed sometimes rolling back builds. Not that your infrastructure is broken. We know this. But application developers, you know, let's not talk about that. Performing database migrations. The best part with all of this is really, we all know how to do a database upgrade. You know you make a backup beforehand, not after applying the alter statements. Not everyone does. It's hard to think about all these things if they're not your daily life. Let's be honest. Operators, it is your daily life, because database migrations do not take Saturday and Sunday off. But codify them correctly so your application teams can use them, and you're good. I think your developers and operators alike will like, in fact, probably love HCP Waypoint Actions. And that's a cool thing. Anytime you are excited about a tool, you use it more. Database migrations are hard any day of the week. If I can make them easier for myself, with the added benefit of making them easier for an application developer, again, winning. And of course, these actions are attached to the relevant Waypoint template, and application developers can execute them right from their HCP Waypoint application interface. There's no context switching for them, necessarily. On the other side, as a platform team, if we're scaling our organization, scaling our deployments, we need to have some more visibility. And that's important. In HCP Terraform, we've got a new feature called the Explorer, which gives you all of that visibility in a very updated fashion. Easy to see what the latest Terraform version is for a specific workspace. What else? Latest module versions, latest provider versions. It allows you to determine what the risk is. Your organization will probably have a policy that says, Terraform can only be one version or two versions behind. Maybe you even go down to the patch level where you say, we have to be at the latest patch level for that version. The Explorer gives you that interface. Of course, I hear you thinking, it's a great UI. It is. 100% agree. It's not just a UI, CSV export, API. So if you need to bring this into Power BI or Tableau to make your product owner happy because they can render a chart from it, totally possible, totally worth it. We talked about this earlier, but I really want to call this one out. Continuous validation is absolutely, I can't say the term kick ass, so I'm definitely not going to say that, but it is a great feature. You get continuous health assessments that help you figure out if your module is behaving the way it should be. And that's important. If I can trust that my code keeps working at a scale, sorry, at a stage where I'm scaling, that means the organization is happy. That means I'm happy. HTP Terraform executes those health assessments in the background. There's no button for you to click. Technically, there's a button that you can say, I don't want them. You should probably not enable that because there's no reason to. But they run, and you will be able to see both in here and in the Explorer how your modules are doing. And that ultimately can be extrapolated to how your infrastructure and your company is doing. You've got features such as drift detection, of course, to make sure your real world expectations match, or your expectations match the real world. And when you're ready for it, you can even connect these notifications to your monitoring systems. So you can move from being reactive to being more proactive. And we're scaling, and that's great. But at one point or another, we realize that this service, this campaign has ended, and we need to get rid of our stuff. So at that point, we set up a workspace for us with auto termination enabled. Just like you might break a glass, eventually, you want to bring down a workspace and destroy it properly. HTTP Terraform will happily help you with that in two ways. One on a time schedule, great thing to do for December 27th when nobody's looking. Or alternatively, when your workspace is no longer being used. We've got explanations on how all of these work on the site, but definitely worth checking out. This is a solid way. If any time you kill infrastructure that is no longer needed, you're optimizing spend. You're reducing cloud waste. It's good for the planet, which is good for you, which means we all win. So HTTP Terraform supports you in this mission, not just because it's important for everyone, but because it's the right thing to do. Any infrastructure you don't manage means infrastructure that can't be targeted. Any infrastructure you don't needlessly have to manage means infrastructure that cannot be attacked. And of course, anything that's connected to HTTP Waypoint through this will also be cleaned up properly. So in the manage phase, we looked at visibility and ongoing health assessments. We zoomed in on the visibility we get with HTTP Terraform Explorer, and we looked at common day-to-day operations. At this point, we're really reaping all the benefits from cost control, risk mitigation, reaction speed. And that's a lot of stuff, right? So if we go back to pretty much one of the first slides I had, if we recap the three stages, we adopted a workflow by making things composable, we standardized on that workflow, we scaled it out. And now we have this thing where it's like, well, we have a design that's good and that should work, but how do we know it makes sense in the greater world? Well, you're not the first one to run into this problem. We've worked with our customers over the past years to figure all of this out. And the result of that is HashiCorp validated designs, guides to scale and operate Terraform Console Vault Packer at scale in that stage three that we defined earlier, accessible to our customers, and of course, continuously expanding. If your organization wants to go from one stage to the next, these guides will help you with that. And I'm super excited to see more of them come down the future. With that, that's all I have for you for today. Part two is coming up with that. You've been a great audience. Thank you.
