Transcript
Testing. I am here together with Jona. Hi, Jona. Good morning, everybody, or good afternoon if you're on Maurice's side of the pond or maybe good middle of the night if you're in the other room. Exactly. Well, good day. Let's just stay with good day. It always works. Yeah, we were talking today about CI and testing for everyone who is doing a hackathon. It's not per se the Veeam community hackathon, but it can be applied to the Veeam community hackathon as well. Or even just coding in general. Yeah, for sure. Because CI and CD is just required in every pipeline, in my opinion. Well, our CI and CD, because I don't think we cover that in our later slides. Well, CD not really, but yeah, testing is important. So we're going to talk about why CD and testing matter in hackathons and in general, but for now we're focusing on hackathons. Source control and team workflows, building your first CI pipeline, testing strategies to fast delivery, practical hackathon tips, a Q&A, and then we are going to talk about upcoming events. So Jona, do you want to take away with the first section? Sure. That will be why CI and testing matter in hackathons or really matter in general. So first off, I'll say what is CI? CI is continuous integration. So essentially automated testing and validation that you can run on your code. And typically you'll see this happen on things like pull requests or sometimes even on every commit, though in some cases that can be a little wasteful and we'll get into that a little bit later. But the idea is continuous integration helps you find latent bugs. It helps you validate your code and the intended behaviors and helps keep your code clean and formatted well when used properly, which means faster output, better code readability, and overall better performance for your code, which especially when you get multiple people working on the same code and on a short deadline can be major challenges. You'll find that bugs are more likely to happen with more people involved, other last minute surprises, it could cause demos to fail, and the last thing you want to do is spend a lot of manual time debugging to fix some sort of code regression or bug that you have unintentionally introduced. So CI is designed to mitigate all of that and reduce your code review time. Do you want me to go to the next slide or do you want to talk about this one? Next slide, please. But yes, so as I said, CI really means every time your code changes, especially before a pull request and a merge into your main code branch, it gets checked, it gets tested, it gets validated, so that hopefully you find those problems in the pipeline as opposed to demo day or even production day. So like I said, when done properly, you get cleaner code, you get faster feedback, safer collaboration, better visibility into where your issues are for debugging, and the ability to move and ship your code faster, which especially in something like the Veeam Community Hackathon is important because you only have a handful of days to design and build your product. Exactly. So now we're going to the next section, that's source control and team workflows. Yeah, source control for the Veeam Community Hackathon, you will get a GitHub repository for your team and we're going to focus on GitHub, but in theory, CI can be done in several different platforms like the GitHub Actions or your GitLab, your Jenkins, for example. There are several different ways to run your workflows. So for us, GitHub is our foundation. We start there because within GitHub, you can create really quickly, really easy, your actions and doing your testing during your pull request. So you need a shared repository, that's what you get already. You want to set up a branching model, make it work with branching and do not commit to master every single time. That breaks things and that will make it harder to read down the line. Work with pull requests, that's with the branching and have a commit discipline. Do not say final, final version two, final fix version six commits or test, test, test commit or dot as a commit message. Make it your commit messages useful. So you have your main branch, that's your stable branch. Everything in main should be ready to work, should be something that you can deploy in theory. You have your feature branches, that features, yeah, that's where you work, that's where you make the changes. So you're doing a feature for API authentication, for reporting UI fixes and so forth and so on. So you make your feature, create a pull request, CI runs, merge. Yeah, and it may not even be a merge domain. You might see a merge to a develop branch or a release candidate branch and then eventually when those are fully ready to ship as a proper release, then those get merged to your main or master. Exactly, but keep it simple. Do not make it more complex than it actually has to be. Like I said, your commit habits, make sure that it's a good commit message, not final, final v2 fix real final. Nobody can read what's going on with that commit, what happened and what changed that commit. If you make a commit like added API out, fixed no response handling, added restore validation test and so forth and so on, that makes it way easier for everyone to read what's going on in your commits. And I will say AI writes some really good commits if you ask it to look at what changes you've made and write your commit message for you. Exactly. That's really, really powerful and can even make a whole wall of text, but keep it small for your own, for everyone else. Make commits small. Do not put like 600 files in a single commit. That makes it unreadable for everyone. Have clear messages like added API auth to your commit messages and make frequent pushes to your upstream branches, like your feature branches that you have on your kit repository, so it's not only on your machine and everyone who is working in your team can read what's going on with your branch and what you're doing. And maybe even steer you. No, no, no, this is not the idea that we want. You should go that way or you should go that way if they do not agree. So pull requests help you refueling the code from your peers. Jonah makes a commit, makes a branch, makes a commit, and I can refuel his changes and say no, this is wrong or this needs to be changed and this needs to be that or oh, that's a really good idea and so forth and so on. Or okay, John's already implementing this over in this other feature branch. Yeah, for example, it can catch mistakes. It triggers your CI. It improves your discussions on your changes and yeah, concurrent feature development. Even in a hackathon, PRs are worth it. It prevents chaos because if everyone commits to main, chaos will occur. Well, and if everyone works on the same branch too, you're fighting over commits and rebases and different things and it's a mess. Exactly. So, Jonah, do you want to continue with building your first CI pipeline? Yes. So building your first CI pipeline, high level, what do you typically see as a workflow and what's included in a mature CI work pipeline? Typically, the developer pushes code or opens a pull request or both because sometimes a pull request, you have to push some commits to fix all your CI tests before you're allowed to merge. The pipeline will pick up that code. It will start. It will build and install any dependencies. Maybe it builds a Python virtual environment. Maybe it builds a Docker image. Maybe both. Maybe it builds front-end assets off of a JavaScript file. And then typically what you'll see is your fast and easy tests for latent bugs run first. Your linters and your type checkers, essentially, which really makes sure does my code follow good standards and even to an extent does it do anything that could potentially introduce security issues or headaches down the road, especially in a language like JavaScript or Python. If you're familiar with object-oriented program like on the Java side, you might be used to where you have an object, maybe the object has name and email because it's a person. And if you try and access some part of that object that doesn't actually exist, it will tell you right in your development session, hey, this is an issue. You're not necessarily going to see that with something like JavaScript or TypeScript without a type checker or a linter. Or maybe you're trying to combine a number with a text string. Those are the sort of latent bugs that those will very quickly and easily check for you. Once those all pass, then typically there's a round of other tests run, either unit tests, mock tests, live tests, a combination. And what those will do is actually verify that your code is behaving as intended. You know, if you're trying to create a function to add two numbers together, then you might have a test and we'll have an example in a bit that says, hey, if I put in 1 and 2 into the function, do 1 and 2 equal 3 in the output? And then from there, especially if anything fails, typically something called build artifacts will get uploaded, which are typically your logs and test results. So you can debug and fix any issues that occur. And then once those are uploaded, it's all published. That way you know, hey, the test succeeded. It failed. Here's my logs. Depending on your pipeline, a build artifact can also be a completed application for your mobile or the web application or Docker container or anything like that. Really, a bare minimum, it can be anything that you want. But typically, correct, it's sometimes the outputted compiled code, but it's also usually your logs from your various pipeline stages. And this all happens automatically. Correct. You just sit back and let it run. So a kind of GitHub action structure, like we said, you have your trigger, you say when it runs, you have your job, which is your collection of tasks, or I'm saying stages because I primarily use Jenkins. But I believe GitHub actions calls it tasks. They're based into your steps, which are your commands. Just breaking it down into small pieces. Multiple steps make up a job, and your trigger kicks off the job. So I think we have an example pipeline of what would kind of be a minimal GitHub action on the next screen. So you can see we've named this test CI. It runs on push and on pull request. So any time code is committed, any time a pull request is opened, and we have a job called test. And in this case, we say it runs on Ubuntu latest, because that's essentially a containerized image or a runner that GitHub gives us. So we're saying run this on Ubuntu. You can theoretically run it on a wide variety of machines, different Linux distributions, Linux with graphics cards, your own self-hosted runner, Windows, depending on what architecture you're wanting to run your tests on, in case you have specific tests that require specific features or commands. Even on macOS. Correct, macOS as well. And then you can see you have steps. And in this case, we're calling the actions checkout step, which is a predefined step or it's a predefined action, because not only can you build your own actions and run, say, your own script, as you can see in the next step, where we're saying run tests.bash script, but there's also community and even GitHub defined actions that you can run. So, for example, here you've got checkout, or if you're using something that requires an NVIDIA GPU and CUDA drivers, there's a community tool out there that you can run a step with a simple line or two to install the latest NVIDIA drivers in the CUDA toolkit. So, what else can CI do? CI can build your containers, deploy your environments. Though really, this is where CI starts getting more into CD, which is continuous deployment. You'll see that it's typically CI slash CD, because technically one is testing, one is deployment, but they're built on the same frameworks. They're just where CI is designed to run on your pull request, CD is usually set to run on a release more often than not. But it can run security scans, it can validate your documentation, it can actually generate your releases. So, you can tag your code, say, hey, this is version 2.5, and your pipeline can go generate a release with release notes with whatever EXEs and compiled software you need instead of just the raw source code. Or it can even publish it to third parties. For example, I run a couple open source projects that we actually use a runner to build a Docker image and upload that image to Docker Hub with tags on release. So, within about two hours of a release being done on GitHub, it's available on Docker Hub automatically for download. All right. And then, Maurice, I think it's back over to you for testing strategies. Yeah, testing strategies for fast delivery. Yeah, testing, that's not the manual testing that you do when you're developing code. No, this is the automated testing. So, what should you test? The core business logic, the API calls, the input validation, the output formatting. And initially, you should skip the SKHs, because they are SKHs. They are probably not going to happen. If you hit one of those SKHs, make a test, but not from the get go. Full performance testing, that takes a long time to do performance testing. And complex end to end setups like testing click, click, click, click, click actions, those take a lot of time to build, to execute, and actually validate that it's doing what you want. If you're doing it for a full workflow for your project, or for your organization, then maybe those are interesting. But for now, during a hackathon, keep it simple. That being said, if you do find an edge case, you should definitely code a test for it. Because this day and age with AI, the last thing you want is, you fix a bug or an edge case, and then someone goes and tries to optimize your code, or add a new feature, and accidentally removes the code for that edge case, and reintroduces the bug, and causes a regression. Or what's this code doing, and delete it, and then things goes out. Right. So, what kind of tests do you have to do? You have to do a lot of testing, so what kind of tests do you have? We already touched on that a little bit. You have unit tests, which is a single function, which is just checking, okay, I've got a function that adds two numbers. I put in one, I put in two, three is coming out. It's a simple test, it's fast, it's just, punch, boom, done. Integration tests is multiple components. You're testing multiple functions in one go. It is medium performance, it takes a little bit more time, but it can be useful. End-to-end tests, it's a full workflow, that's really slow, because you're testing a lot of things. Start with the unit tests, continue with integration, end-to-end tests can go later, because as long as your coverage is okay, you probably don't need the end-to-end tests. It takes a lot of time to create those end-to-end tests. So, for PowerShell, I've created a super simple example. You have got a function testBackup, and the return of that function is per definition true. So, my test is, in Pester, describeBackupTest, it should return true, that's the thing that we have as a result, and testBackupPipe should be true. That's a successful test, because it always returns true. If you say, should be false, it's failing, and it says, okay, it should not be, it's false, it's true instead of false. The example with the function to add to numbers, it's fast, fast readable, because it's really, really, really simple. It says, you are sure that the function add with parameters one and two should be three. For example, if you say return one plus two, instead of A plus B, and you say, let's do three and five, it still returns three, because your function definition is incorrect compared to what you are sure that you will get back from your function. So, one test here is fine. Multiple tests makes it better, because you're testing multiple ways of adding this function. For add, it's doable for some functions, it's not, it all depends on what you want and need to do. Then, failFast, yeah, why? Because it's, it makes it easier to debug and find where the issue is. If you can failFast in your pipeline, you can work real quick with your pipeline to figure out what's going on. Lots of people like to take all their steps in their stage and run them all at the same time, because, oh, that's the fastest way of completing it, but that's not always necessarily true. Sometimes it's faster to run your linters and your type checkers first, then your mock and unit tests, and then go into your full end-to-end and integration tests, because the earlier stages, while it does add time overall to your pipeline, those stages might complete after two minutes instead of an hour, which means you can go back and fix those, and they may unintentionally fix failures in your end-to-end tests, which, in the long term, saves you a lot of time in your iteration. Exactly. Then, we're going into the practical hackathon tips, Jonah. Yeah, well, and building on that and what we were just talking about at failingFast, I'll say it's not going to be on the next slide, but one of the tips I say is find the right tool for the right job, you know? I have been overseeing development of a customer portal for my company, and one of the things I have found is there's a lot of gains to be had by just looking at the tooling that we're using. You know, we use Python, we use TypeScript, and we were using what were very common linters at the time, you know, PyWrite for type checking and PyLint for the linter, but we found by moving to more modern tools, especially ones built on Rust, we were getting more comprehensive tests, and we were speeding up those stages in our pipeline by five or even sometimes 10x, because we were going from a five-minute test to a 30-second test, which meant that we were reducing bugs and we were able to find those issues and fix them a lot faster and run more test pipelines in the same time frame. But into the hackathon-specific tips, keep the scope under control. Don't build too much too quickly, like Maurice was saying. It's really hard not to do, especially this day, with AI and people using AI to help them code, but you don't want to be the guy that submits 600 changed files and 100,000 lines of code in a pull request. Keep the features small and modular, even if you have to do multiple iterations of six or seven phases and merge each phase into your development branch at a time as you build out the feature. It is much easier to review the code in smaller pieces. Exactly. Nobody has or can take the time to review 600 files. That takes way too much time. And if they are, they're not actually doing it. They're taking a different AI and asking it to review for them. Yeah. So using CI as a team signal. This is where, you know, crucially, pull request checks. In many cases, you will see a lot of code open source and enterprise software is you are not allowed to merge your code until your CI is green across the board, because you don't want to introduce any issues that could cause headaches down the road. Yeah, but also make sure that your coverage is going up and not going down. Right. Well, that's another thing we haven't touched on in here too, right, is that's another stage is coverage checks. Maybe you set a minimum threshold of, I want 75% of my code to be tested by these mock tests. That way, because the more your coverage goes down, the less of your code you're actually testing and the less effective your CI is, which means the more likely that you find bugs in demos or production. Yeah, it's better to create a test first and then write your code and write your code and then your tests, because your tests define what you want and your code is built based on your tests. Right. That's called test driven development, by the way, if you want to look it up. And if you miss something, you change your test or you add a test. Exactly. And then documentation is key too. So a readme is always good for a project, you know, what it does, how to install it, how to run it, how to test it, crucially, because just because you build a GitHub actions YAML that says, hey, run this on a pull request, in theory, all of those stages can be replicated locally, especially these days with AI, right? Like all those type checkers and linters that I keep talking about, you can put in a Cloud readme that says, hey, before you push any code or complete work, run these couple of checks locally, which is not going to 100% catch things all of the time, but it will catch 99% of the common issues that would otherwise ship to your pipeline and slow you down in running through multiple tests and iterations to fix it all. So the final checklist, like we said, your code in your repository, a complete readme, and I'll take it a step further. Plus AI readmes. So your Cloud.md, your agent file, whatever you use, if you're using an AI, have a complete readme updated regularly. Because that's how you enforce code standards on any AI output that you do. Once you've got your code and your readmes in place, is your pipeline passing? Do you have the right tests included? Do you have a demo ready? And is your team aligned on what you're working on? You know, especially working on a team, you know, Maurice and I could both be working on two completely different things. But there might be just enough overlap that we need to close different things. But there might be just enough overlap that we need to collaborate where that little bit of overlap is, or else it can cause merge conflicts and introduce bugs and other issues. Like maybe Maurice is working on new dashboard widgets, and I'm working on rate limiting and bot prevention. So Maurice needs to make sure that he's not making excessive calls to generate those widgets so that he doesn't trigger my rate limiting and lock a user out of the system. Yep. So as a wrap up, CI reduces risk. Testing saves time down the line. Source control enables teamwork and small habits create big wins. It's, I'm just reading what's on the slide, but it actually helps. Now we have one more thing. Questions. Are there any questions? If you have a question, you are able, you're allowed to unmute yourself or write them in chat if you don't feel like speaking. I'm not seeing any messages coming in or anyone unmuting, so I'm gonna assume that there are no questions. So as a reminder, the fourth annual Veeam Community Hackathon is on October 12th, 9 a.m. Eastern time till October 16th, 9 a.m. Eastern time, or 1500 Central European saving time. Signups are open right now until October 1st. If you're not already joined, scan this QR code with this cute alpha in the middle. And yeah, Sophia is clapping about it, but it's Alfred. So yeah, everyone like it, right? And signups are still open until October 1st. Please join. Upcoming events. July 28th, we will be doing a spotlight. August 13th, if I remember correctly, we will be doing a Veeam 100 special edition. And on August, sorry, August 13th, not October 13th, why don't we? August 13th, we'll be doing the special edition. August 25th, we'll be doing an overview of AI coding tools. I'm really looking forward to that one because I think as of today, AI is here with coding, and it's becoming more and more important. On September 29th, we're going to do another spotlight. And then in October, we have the hackathon. Again, please join. And then, Johanna, if you don't have anything else to add, I want to thank everyone for your time and participation in this webinar. Took the words right out of my mouth. And join the hackathon. Thank you.
