GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker

[00:00] In this GitHub Actions tutorial we will go through the following topics. First I'm going to explain what GitHub Actions actually is and we're going to look at specific developer workflow use cases that you can automate with GitHub Actions.

[00:14] After that I will explain the basic concepts of GitHub Actions including the events and actions and workflow and how GitHub Actions actually automates these workflows using these components.

[00:26] Having understood what GitHub Actions solves and how it makes it possible, I will go through the most common workflow which is CI CD pipeline. I will explain shortly why it's not just another CI CD tool or what are the benefits

[00:41] of GitHub Actions CI CD pipeline. Finally, we will go through a hands-on demo where I explain the syntax of GitHub Actions workflow file and then we will go through a complete CI pipeline setup with my example

[00:55] Java Gradle project which we will build into a Docker image and push to a private Docker repository on Docker Hub. So what is GitHub Actions? GitHub Actions is a platform to automate developer workflows, so software development workflows.

[01:14] Many of the tutorials that I've seen seem to convey that GitHub Actions is a CI-CD platform, But as I said, it's for automating developer workflows and CI, CD, Python is just one of the many workflows that you can automate with GitHub Actions.

[01:30] So now we need to understand what are those developer workflows. In other words, what is that the developers typically do that is so time consuming or error-prone or just tedious that it needs automation.

[01:46] So let's go through a couple of specific examples. Now as you probably already know, GitHub is a platform for a lot of open source projects. So a lot of developers who have developed their own libraries for Java or some other

[02:02] programming language, they can host their projects on GitHub and make them publicly available as open source projects so that the community can use those projects but also

[02:14] to contribute to those projects. So when a team or individual developers who manage those projects can be contributors or things happen inside the repository, people creating pull requests, people joining in

[02:29] as contributors and so on, they have a lot of organizational tasks to manage. Let's see examples of such tasks. Let's say you have created a library that makes it easier to work with daytime in Java.

[02:42] So it's a Java library that you created and you have some contributors and users of that library. And whenever a user of the library sees that a new release of the library has a bug or something

[02:55] isn't working, they can create an issue that something is not working. So you have to check that issue, you have to sort it, is it minor, is it major, is it something urgent, is it something that others may have also reported, is it reproducible

[03:10] for example. Maybe you assign it to one of the contributors or to yourself and so on. Let's say one of the contributors fixes the issue and creates a pull request so that you can merge it into the next release of that library. So you look at the pull request, you review the code,

[03:27] you make sure that the issue is not reproducible anymore with the fix, and you merge it into the master node. So this is going to be part of the next release, so to say. So you want to release

[03:40] the next version so the people who use the library can upgrade the version where the issue is fixed. So after the pull request is merged into the master branch, you want to start a pipeline, a build pipeline that will test your code, build your artifact, and so on. You also want to maybe

[03:57] prepare some release notes where you document what got added in the new version and maybe also adjust the version tag or the version number. So all these things are workflow examples of what you

[04:11] have to do as a maintainer of such repository. So you can imagine the bigger the project gets and the more contributors you get and the more features and issues they fix and more pull requests they

[04:23] create and the more people use your project, the more organizational effort it is going to be. So obviously as a developer you don't want to be sitting there doing all this tedious organizational

[04:35] stuff. You want to automate as much as possible of those management tasks so that you can also concentrate on programming and developing new features and new functionalities in the project.

[04:47] And for that purpose GitHub Actions was created. So with GitHub Actions every time something happens in your repository or to your repository, you can configure automatic actions to get executed

[05:01] in response. And these things that are happening in your repository or to your repository are events in the GitHub space. So someone creating a pull request is an event, someone creating an issue

[05:15] is an event, someone joining as a contributor is an event, or you merging that pull request into the master branch is an event. Also note that other applications or tools that you may have

[05:29] integrated into your GitHub can also produce such events that you can respond to with automatic actions. So when you automate these flows, basically the concept is pretty simple. You

[05:41] listen to any such events and depending on what event happens, you want a certain workflow to execute automatically. So every time someone creates an issue that's an event, maybe you want

[05:55] to automatically sort that issue, maybe label it, assign it to a respective contributor, or maybe assign it to you per default, maybe categorize it, and also maybe write a script or a test that will

[06:08] try to automatically reproduce the issue. And then add some status or comment or something that says reproducible or non-reproducible. So all these things can be automated with actions. So each

[06:22] small task that you automatically trigger on an event is going to be a separate action. So writing a comment, putting a label on an issue, assigning it to someone, etc. Those are actions. And the

[06:37] chain of actions or this combination of actions actually make up workflow. So now that we understand the basic concepts of GitHub Actions, let's look at a specific workflow example.

[06:49] So obviously not everybody has an open source project on GitHub. You can have your own private projects of GitHub for the application that you are developing. So the most common workflow you

[07:01] will think of for your repository would be a CI-CD pipeline. You commit your code, the build starts, it tests your code, builds it into an artifact, then pushes the artifact in some storage,

[07:14] and then deploys the application on a deployment server. Now why is it a big deal to have just another CI CD tool? Well the first and obvious advantage is that if you're already hosting your

[07:28] code on GitHub, now you can use the same tool for CI CD pipeline as well. You don't have to set up another third-party tool and manage it separately, you have that integrated into your code repository.

[07:43] Another advantage of GitHub Actions that I see is that it makes it really easy to build your pipeline. So the setup process of the CI-CD pipeline is really easy. It is actually meant to be a tool

[07:55] for developers, so they made it so that if you have a developer's team, you don't need an extra DevOps person who is dedicated to setting up and maintaining that CI CD pipeline in your project So now the question is how do they make it easy or how does it compare to other CI tools like Jenkins for example

[08:15] and why is it much easier to set up and manage? So you know that when you think about CI-CD pipeline one of the most important things is its integration with different tools. So what do I mean by that?

[08:29] Whether you are developing a Node.js application which will be built into a Docker image and then pushed into a Nexus repository and deployed on DigitalOcean server or you're developing a Java application with Navin, you have

[08:44] integration tests to test your application on Linux and Windows servers then build it into a Docker image and push it to AWS Container Registry and

[08:56] and deploy it on AWS EKS. So basically you can have many different combinations of tools that you're using in your development process. So you don't want to be sitting there trying to configure

[09:10] your CI CD pipeline with all these tools, like installing Java and Maven and Docker and all these integrations with Nexus and AWS and so on, installing plugins and configuring them.

[09:23] Instead, you want a simple way of saying, hey, I need an environment which has Node and Docker both available without me installing any of it with a version that I want.

[09:35] And the same way I wanted to do the deployment part easily by simply connecting to the target environment and deploying the application there. And that's exactly the advantage that you have when you're using CI-CB pipeline in GitHub Actions.

[09:51] And of course, I will show you and explain how this works in the next demo part with my example Java Gradle project, which we will build into a Docker image and push to our private Docker repository.

[10:06] So to see all this in action, let's go to GitHub. In here, we can create a test repository. Call it my project. Let's make it public and that's it. So this is my empty project, so to say.

[10:20] So whenever you create a new project you have this Actions tab integrated into the project that lets you get started with automating one of your workflows.

[10:32] So now I can push my local code to the remote repository. So let's go back and refresh it. And here I have my Java application which uses Gradle project.

[10:45] So let's go to Actions. So here if we scroll down, we see a big list of workflow templates, which means you don't have to start writing your workflow file from scratch. You can use one of the templates that matches technology your project uses.

[11:01] And these are actually grouped in three main categories. Here we see the deployment workflows to deploy your code to cloud services or using some automation tools. And here we have a big section of continuous integration workflows.

[11:17] And here if you look at the list, a lot of options based on what programming language you're using, what tools you're using, and also combinations of such tools. So for example, you have Java with Gradle, and you also have Java with Maven, and so on.

[11:33] So you have the build and test workflows as well as publish workflows where you publish your artifacts to some repository. And that's where I was talking about when I mentioned that different applications use

[11:46] different combination of tools and it's important for a CI-CD tool to have an easy integration with many different tools so that it works for different projects. All the way down, these are the workflow examples that I mentioned at the beginning, like greeting

[12:02] someone, for example, if a contributor joins your project, you might want to send an automated greeting message to welcome or labeling your issues and so on.

[12:14] And obviously you can make your own workflow with different combination and adjust it. So let's create a build workflow for our Java Gradle application. And obviously I will choose the Java Gradle workflow template and let's see what the workflow

[12:27] file looks like. And see what happens. It automatically creates this configuration view in my project or my repository. It creates this path.github.workflows and this is the file that basically holds my workflow

[12:46] logic. It is written in YAML format. And what's great with this list of workflow suggestions is that you get a pre-considered workflow that you may need to adjust just a little bit, but most of the stuff is already

[13:02] here so you don't have to start from blank file. So let's go through the syntax of this workflow file in detail so that we understand how to write our own workflows. So I'm going to copy this in my editor so we can see better.

[13:18] So first of all we have the name of the workflow. This is basically for you to describe what the workflow is doing. These are the events that are mentioned. So every time an event happens or events happen, we can trigger a certain workflow.

[13:36] So this is a section where we define events that should trigger the following workflow. And I think it's pretty intuitive. Every time someone pushes to master branch, we want to trigger this workflow.

[13:49] Or every time a pull request gets created with master branch as a target, this workflow will get executed. which in this case makes sense because every time something gets pushed into master or you want to

[14:01] merge something into master it makes sense to run tests of the test or application to make sure that it's mergeable so to say or that we didn't break something in the master branch. So that's pretty

[14:15] straightforward. Other examples that I mentioned could be creating an issue or a contributor joining. this will be all events listed here. You have a complete list of such events

[14:27] documented on this page. So here you see the list and here you see some more detailed explanation and also the usage. And as always I will put all the relevant links for this tutorial in the video description. And this is a part that gets

[14:42] executed whenever these events happen. So you have jobs, this is the names of the job this should be arbitrary just like the name of the workflow so you can name

[14:54] it yourself and job basically groups a set of actions that will be executed right so as I mentioned events trigger a chain of actions or a combination of

[15:06] actions and these are defined here so let's look at the first one pretty logical whenever we want to build application or run tests we need to check out the repository or the code first. So how does this get executed or

[15:23] what is behind this syntax? Actions path in GitHub is where pre-created or predefined actions are hosted. So basically you can assume that everybody

[15:35] who uses a CI CD pipeline in GitHub actions will need to use checkout command, right? So instead of letting everybody do that on their own, they're creating an action called checkout that people can use. So if I go to GitHub, slash actions, I will see a list of

[15:55] repositories that contain all those actions. So let's look for our checkout action. So these are, all these actions are basically repositories. Let's go inside and here you have action yaml.

[16:10] So this is a normal repository with some code in it and we have action YAML file here This is basically what checkout action does in the background or the logic that

[16:23] people already wrote so you don't have to write it in your workflow and just reuse it. In each action in the repository we have some sort of documentation where you see if you can configure some additional parameters for

[16:38] this checkout action. And this here is version of that action, so to say, because as I said, these are simple repositories that are built and released and have versioning. So this is our checkout

[16:53] step and whenever you are using an action that is already available either at slash actions or maybe some other community or team has created one, you can basically use

[17:06] any such action using the uses attribute. So these are the official ones, but whenever someone creates an action, basically a repository with action.yaml file, you can use it here

[17:19] using the uses attribute. So the second one, second step, is action called setup.java, which is another repository in this actions list, and what it does is basically prepares

[17:34] your environment with Java with a version that you defined here. And this is the part where I mentioned that you don't have to install or configure anything like in Jenkins for example you would go and configure your Java version. Here you define that you want to use an environment

[17:52] with Java on it. So Java version 1.8 will be installed and available there. The next one is a command. So here you see the difference whenever we are referring to action in a repository,

[18:07] we use this attribute. Whenever we're running a command, just like a Linux command for example, a command line command, we are using run attribute. So this basically just changes the permissions

[18:20] of gradle file and the next step just calls gradle build. And all this is done in the same environment, So your code gets checked out, the Java version gets installed, and then you call Gradle build

[18:34] in the same environment. So obviously for this to work you have to have a Java application that is built with Gradle. And now let's actually go ahead and execute this workflow for our Java project.

[18:47] The name of the YAML file is also something that you can decide for yourself. We can actually call it CI and start commit. Let's create a new branch and create a pull request that will

[19:03] be merged into master branch. And here you see that the workflow got triggered because our event matched to what just happened so we created a pull request against master branch

[19:20] and that triggered the workflow. This is in progress and if I go into details, we're going to see what is actually happening. So we'll complete it. So let's actually look at the steps that we are executing.

[19:35] Setup job, which basically prepares the job environment for executing the workflow. Here you see, for example, these action repositories got downloaded so that it can be used.

[19:47] Here you see the checkout action And you actually see pretty helpful information in all these steps. And they also highlight the command that gets executed so that you can easily see, first of all, the command

[20:01] and differentiate it from the log and also see how they can interpret your command. For example, with options and flags and environmental variables and so on. Then we have the setup JDK.

[20:14] Again, this is the command that got executed and some log files. This is where the build, the actual build happened. Build successful. And then we have some post build actions which we didn't define.

[20:26] These are auto blocks. Things get cleaned up. So in my opinion for an initial setup of a workflow or such a workflow it's actually pretty straightforward and easy to set up and it's pretty difficult to mess this up.

[20:41] So now you may be asking where do all these things get actually executed? Because you see that the code got checked out, then you see some commands got executed, the Java version got installed and the Gradle build actually happened.

[20:55] So where do all these things happen and how do they get executed? So the way it works is that workflows on GitHub Actions get executed on GitHub servers.

[21:07] So it's managed by GitHub. You don't need to set up some servers and configure your build tool, install some plugins or whatever and prepare it for building the application. So GitHub will manage all of these for you.

[21:21] The servers will be configured and ready to execute your jobs. An important note here is that whenever you create a new job or whenever you create a new workflow with a set of jobs,

[21:33] for every single job, a fresh new GitHub server will be prepared or used to execute all those steps inside the job. So one job will run on a single server at a time. So for example if you have a list of jobs here,

[21:53] maybe you have a job that builds the application and then you have another job that publishes Java artifacts let's say to a repository. So one job will run on one server, another job will run on

[22:08] another server. By default these jobs will run in parallel, but of course in such a case you would want to wait until the first job was successful to execute the publish. So here I can have a publish

[22:23] job. Of course in this case we want to wait for the build job to successfully execute before we publish the artifacts. So we can override this default parallel execution using needs and we can

[22:37] reference the build that it depends on. And then we'll have a set of steps and actions here. And another thing that should be noted here as well is this line here, runs on. So the servers

[22:51] that I mentioned that GitHub makes available for the workflows to run come in three categories. So you can choose either Ubuntu, Windows or Mac OS. So for example if you have an application that you

[23:05] are shipping out to customers that have all three operating systems, you can test each release for example or each commits to master, you can test that on all three operating systems.

[23:22] And the way we do that is using those attributes. So we have a strategy, a metrics. Metrics is used basically whenever you want to use multiple operating systems or maybe multiple versions of Java or whatever technology you're using for your application.

[23:39] And here I'm going to define OS options as an array. So we have the fontu latest, we have windows latest and we have macOS latest. And here on runs on we're going to reference it

[23:57] list using metrics.os and let's actually try to apply this change. So I have merged my pull request so here you see in the master branch we have this .github.workflows path with the ci

[24:13] memo file inside. So now this has become part of the application code. So I can adjust it here and let actually commit straight to master branch And let see our workflow And here you see three builds are getting executed in parallel on all three operating systems

[24:35] So with next steps we're gonna take that Java artifact file and we're gonna build a docker image out of it. because we live in a world with containerization, so jar file won't do it.

[24:48] So once we have the Docker image built, we're going to publish that to a Docker Hub private repository. So let's do that. First I want to have my Docker repository set up on Docker Hub.

[25:01] It is super easy, actually just create an account, and you get one private repository for free. So this is my private repository, and I just have two images here of different applications, and this is where we're going to push our Java demo image.

[25:17] So what we're going to do is, let's go back to the editor so we can see the syntax highlighting better. So here is the next step, I will add a step and let's call it build and push Docker image

[25:30] because that's what we're doing. And here we have a choice of either running the commands or using an action. So first we can write here all the commands we need for building and pushing Docker image.

[25:45] This will be Docker login with credentials, because first we need to log into the repository from GitHub so that it can push the image there. Docker build, Docker tag, and Docker push.

[25:58] Two notes here. Whenever you want to execute a step with multiple commands, so our own command line command, so to say, you can do that using the pipe syntax. So this is a multi-line syntax in demo.

[26:12] So here we would have Docker login and some credentials here, and we would have Docker build, Docker tag, and so on. And another point is that on a Linux Ubuntu machine,

[26:26] so this one here, we have Docker pre-installed. So I don't have to set up environment, I can execute Docker commands right away. But as an alternative to command, as I mentioned, we can use an action.

[26:39] And since building and pushing the Docker image is a pretty common step, something that a lot of projects will be doing, we can expect an action to exist or multiple such actions

[26:51] to already exist. So what I'm going to do is I'm going to go and find an action that does exactly that. So we can Google.

[27:05] And here you see we land on the marketplace of GitHub Actions. So if I go here, I see a bunch of Actions, and here I can look for different functionalities

[27:17] that I may need in my pipeline. So we can go with this action, and here you see the usage example. So what's important with Actions is that you have a possibility to pass in the parameters.

[27:29] So it basically does all those Docker login, Docker build, etc. commands, but obviously want to set our own credentials. We want to tell which docker registry it should connect to. In this case we have a docker hub registry and of course the

[27:45] image name. So we can actually parse all those as parameters. Here you see it supports multiple docker registries. Here we have the docker hub and here we have the example usage as well. So I'm just going to copy that and we can paste

[27:58] it right here. So this is the action with the version. This has the version 4 right here and these are the parameters that we can override. You also have the

[28:13] description of the parameters and which one does what and what parameters you have available. So let's see which ones we need to override. The docker registry name for Docker Hub is docker.io. We need to set the image repository name and

[28:28] and image name. So I'm going to go back to my Docker Hub and just copy that. So this is the Docker ID and this is the repository name. And here you see credentials for username

[28:46] and password. As I mentioned, GitHub needs to authenticate itself with the private Docker because it's obviously secured. And we need to provide credentials here. And since this

[28:58] YAML file, this workflow YAML file, is part of the code, we can't just put plain text credentials here. So we're using placeholders instead. So these are referencing secret. So where do those secrets

[29:11] come from? These ones actually can be created in the GitHub itself, which is a pretty convenient way to store all your secrets for your repository. So if you go to settings and secrets, here you can add

[29:27] secrets that your workflow uses. So I'm going to post my secrets in here. So this is the username and this is a Docker's password. The name should be of course what you're referencing here.

[29:47] So I'm going to put in the password. So these are the username and password that you use to log in in the Docker Hub right here. So I have my secrets here for my repository. So now I can reference

[30:00] those secrets from my workflow files using secrets.the name of that secret. And note that the syntax right here is the same as here. This is basically YAML syntax for

[30:12] referencing values. And for this example we actually don't need all three operating systems. Let's go back to Ubuntu. Also, as I mentioned, Ubuntu is the one that has Docker pre-installed.

[30:26] So this will be basically our step for building and pushing Docker image. And I think this is more convenient because I don't have to write out all these Docker commands for building and pushing the image.

[30:39] It all happens in the background. You can also override the tag name here. You can override the location of Docker file, but default is just the current directory. We're going to leave this as default. So let's actually go ahead and execute this step.

[30:53] I'm going to copy the whole file and let's replace that. And let's start our commit. I'm just going to commit it straight to the master branch.

[31:05] So here is our new file and if I go to actions, here I see that the workflow ran and it completed. So let's actually look inside the steps.

[31:17] Here we have build with Gradle and here we have build and push Docker images. Let's actually open this one up. Seems like everything executed just fine, it was successful. So if I go to my Docker Hub repository and refresh, here I have a new tag of my Java application.

[31:36] And this is the default tag that this action gives my Docker image, which has a branch name as a prefix. And as you saw here in the parameters, you can actually override that tag as well with the tech program. So that was our continuous integration workflow. We built Java

[31:53] artifacts, we built a docker image and we pushed it to docker repository. I will make a full course on the GitHub actions for multiple other use cases including deploying the docker image on cloud or Kubernetes environment, testing and

[32:08] building with Node.js application as well, and also automating some other workflows. So if you're interested stay tuned for that. So hope you learned a lot in this video. Let me know your feedback also what else you would like to see and learn

[32:22] on this channel or if you have any questions in the comment section below. Thank you for watching and see you in the next video!