CI/CD Pipeline Using GitHub Actions: Automate Software Delivery (for free)

[00:01] so in today's video we're going to look at cicd pipelines and in particular how we can create them for free using GitHub actions CI CD is not something I've going to quickly explain what it is and why we use it so CI stands for

[00:15] continuous integration and it's the process of automatically running all of your unit tests and integration tests for each of your builds now you don't integration tests running on every single commit it can just be whenever

[00:29] you create a pull request your test run automatically and at least this way if the tests fail you can stop that code being merged into develop or Master you your unit tests on your machine before you raise your pull request but having

[00:42] server is a good quality control to have in place and it also prevents the running on your machine and not anyone else's in the past I've used tools such as teamcity and Jenkins to run my CI pipelines but more and more teams are

[00:58] already on GitHub then it's a bit of a no-brainer to run your tests on there as well and it will also save money on licensing and server costs now CD can continuous deployment they're both quite

[01:12] continuous delivery is really about building and packaging up your on the server but it doesn't actually include the deployment of your application continuous deployment on the other hand also covers the deployment of

[01:27] environment most companies don't like having applications going straight to production without any manual intervention it's quite typical to have the dev and QA environments but when it comes to production generally you want

[01:41] some manual controls in place in the past I've always used octopus deploy to different environments but it's actually possible to do that in GitHub actions as functionality of GitHub actions I've created a very simple.net core API that

[01:57] writes to a mySQL database this project include unit and integration tests and those as well as show off the results in a nice format for the continuous to have GitHub actions build out a Docker image and have it push it to AWS

[02:12] ECR for me I'm not going to cover how to do any of those manual approval a future video all the code that you're going to see in this video is available to my paid newsletter subscribers my newsletter is otherwise free at the

[02:25] channel then that's the best way to do it I'll leave a link in the description so you can check it out so the dotnet core API that I've put together for this project is a very simple library application so think something similar

[02:37] to Goodreads so this application is really made up of three layers we have our API layout which has all the controllers all the endpoints and handles things like validation we then have the service layer that handles all

[02:49] our persistence layout which is saving everything to the database I also have an SDK which I've written using a great.net core package called refit and this is what I'm then using in my integration test to be able to call my

[03:02] API so I've got some endpoints for adding authors to the database as well as getting them either singularly or as a paged result I then have another controller that lets you add books as well as retrieve them in a similar way

[03:14] got unit tests I haven't done a complete set but there's unit tests for both the service layer and then for the that show that you can create an author and retrieve it as well as creating a

[03:29] integration tests aren't using any of the API code they're just relying on the SDK so in order to be able to run the integration tests in GitHub actions we the database and the way we do this is with Docker so because I've built my

[03:43] application with net core I've put together a multi-stage Docker file to be able to build and run my application now there's probably other ways depending on application and GitHub actions but I'd recommend if you're going to be

[03:56] deploying your application using Docker anyway that you use that Docker image for testing your application so there's a few things that make up a net or Docker file we have first we're going to use the SDK image from Microsoft so this

[04:08] application we're then going to copy across all of our project files and this against them now the reason that we separate this instead of just copying across everything is because.net restore is quite a lengthy process you've got to

[04:22] download all the new get packages and by separating it out we can basically have in Docker so provided you don't change any of these project files by adding in any additional dependencies it means that when you run.net restore it

[04:36] run it again even if you change the other files in your application once.net restore is done we then copy across all the other files and then run.net publish in release mode to be able to build our application now of course we don't want

[04:49] part of our Docker image because it's going to make our Docker image quite large we're then going to get the.net runtime image and we're going to copy built in the other container and you can see here I'm exposing Port 5275 which

[05:03] we're then going to use later on to call our API so as I mentioned earlier to be need a database running in the back end so I'm using MySQL for this and I'm just using the standard MySQL image so you can see here in my Docker compose file I

[05:17] have a database service set up using MySQL 8 and I have some very basic and not very secure passwords set up here just for use for testing and then I'll have our API set up and you can see here

[05:30] files in the same folder and then we're tagging this image with GitHub actionsdemo.api and then version and I'll cover where that version is coming from in a bit so you can see here I've got the port set to 5275 by then mapping

[05:45] that 5200 in the docker compose file and this is so you can run the docker compose file with the database and not worry about any port clashes when you run the API locally on your machine so if we run Docker compose up we can see

[05:57] this is going to spin up our mySQL database as well as our API as well we be able to call IPI and make sure that everything works for the paid project I've also included this Postman collection as part of it so you can

[06:11] easily test out the API so that's how this API works next we're going to have see how we can put everything together so we can run our tests in GitHub so GitHub actions does have a very generous free tier you can run the standard Linux

[06:25] Runners for free if you've got a public repository for private repositories you you can use for free this is only for the Linux Runners if you opt for the windows Runners then you're going to use twice as many minutes and if you opt for

[06:39] use 10 times the amount so you're going to see the amount of three minutes you have go down pretty quickly so for this example I can use Linux as.net core as on what you're using so the first thing we need to set up is a name so I've just

[06:53] called this build test and push we then specify when we want this to run so just for this example I'm doing on push so every time I push something to my repository it's going to run this test but you could set it up to just do it on

[07:05] pull request or on merge for example and you can even specify particular branches that you want it to run on now we need to set up our jobs for this example I'm just setting up one job called build so you can create multiple jobs in GitHub

[07:17] actions but it gets a bit complicated you can't just use variables and data that you've had in one job to be used into the next job you have to use the artifacts and upload them and download them between each of the jobs again I'll

[07:29] but for now it's easier just to keep everything in one job next we specify where our job runs so this case I'm just using Ubuntu latest again this is the cheapest option if you're using GitHub actions so first we need to check out

[07:42] our code now GitHub obviously comes with a standard action to be able to do this part of this setup I want to get the semantic version of my application to be able to tag my Docker image and to do that I'm using Git Version but I found

[07:55] Git Version doesn't actually work unless you go and grab the tags using git fetch fetch an unshallow copy of my code I can then install Git Version and then run Git Version execute to be able to get the semantic version of my code next

[08:10] we're going to set up.net as I'm using.net and here we specify the version so I'm using.net7 we then need to run the same commands that we would on our machine so in this case we run.net restore to build our

[08:23] dependencies and net build to build our project so now we've got our.net code built on the GitHub Runner so next I'm going to run my unit tests so I'm running.net test and I've got a filter here with a category equals unit now I

[08:35] can do this because I'm using X unit and I've set up a trait to specify the category so when I run my unit tests this is going to produce a test results file and this is because I've got this vs test logger set up in my project now

[08:48] I've set this up so it will put all the test results into a test results folder inside the test project and then later I'm going to use the test reporter to be able to show these test results in GitHub so I I have to create a test

[09:01] results folder in my root project and copy across the tests results now the reason I'm doing this is because when I come to run my integration tests it's going to overwrite the unit test results for some reason when you use a filter it

[09:13] produces the test result file which is a bit annoying so now that we've run all integration tests now to be able to do this we need to spin up a Docker container and we can do that using our Docker compose file now you can see here

[09:27] specifying that we should always build our project and I've got the minus D's can see here I'm specifying a version earlier in our Docker compose file and this is going to get the nuget version

[09:41] and this is going to make sure that when we build our application in Docker with the correct version now in The Next Step we're just going to sleep for 30 do this you could call the API to see whether it's up and running but this is

[09:56] reason we have to do this because if we jump on onto running our integration containers might not be up yet and it will just fail now for our integration tests we're just running.net test again because they're just X unit tests and

[10:10] this time we're filtering by category integration and here I'm specifying the that it knows to call the API that's once that's run we're going to copy across the integration test results into

[10:23] our test results folder next I'm using this test reporter action from the GitHub actions Marketplace this gives you a nice view of all of the tests from your application and whether they pass failed or been skipped and how long each

[10:35] about this test reporter GitHub action is it also works with lots of different languages it works obviously with net we've got X unit and unit and Ms test but also works with dart flutter Java and JavaScript so if you're using any of

[10:48] those you can get a nice report from your test results and you can see here we've got if success or failure which basically just means this is always integration tests passed or failed now for the continuous delivery part of this

[11:01] workflow I'm going to push the docker image that we previously created up to AWS ECR now I've already gone ahead and created an AWS ECR repository and I've key and a secret key now to make sure that these Keys can't be seen I'm using

[11:16] the GitHub secrets that stores them encrypted in your repository and then they don't appear when you build now I've created another if statement here which means that this step is only going to run if you're running on the main

[11:29] branch if we run it on develop or on a pull request Branch it's not going to push anything up to AWS ECR if we run all this on GitHub actions you can see it all goes through and passes we've then got a nice test report that shows

[11:41] our unit tests on our integration test results and finally if we look on AWS you can see here that the image is uploaded successfully now if you want to project if you become a paid subscriber of my newsletter you'll get complete

[11:54] to all of my future videos and I'm also planning on releasing a few courses in generous discounts for all of my paid subscribers thank you for watching this video I hope it's been helpful and I'll see you in the next video