Local development with Kubernetes And Jenkins

For the next part of the series on JobRM we will set up Kubernetes and serve the static website from the Kubernetes cluster. The whole process will be automated as much as possible. We will create the environment with Jenkins and use a Helm chart to deploy our application. This will all be done on our local minikube server. Before getting started, you will need to install minikube. Next, configure and hopefully save your minikube start command somewhere; I will put mine in the repositories ReadMe.

minikube start --vm-driver=hyperv --hyperv-virtual-switch=Minikube

Since I am using Hyper-V for minikube, I will need to stop the cluster and remove the dynamic memory allocation setting. The official documentation is not very helpful but is fairly simple. Once minikube is started and installed, call the minikube stop command and go to your Hyper-V manager. Right click on minikube virtual machine, go to Settings... and Memory and uncheck the Enable Dynamic Memory checkbox.

Once the machine is up and running I use minikube start and minikube stop to manage the cluster. At this point, initialize helm with helm init and wait for Tiller to be available on the cluster. Quick check with kubectl version and helm version should confirm that everything is up and running.

Jenkins

Once we have our cluster up and running and will need to get something that can build and deploy our application. For that we will use Jenkins and the pipeline feature to manage our workflow. A great starting resource for getting Jenkins running with Kubernetes has been from lachie83/croc-hunter.

One of my goals is to be able to develop most of the pipeline for the build locally without having to create a production Jenkins and hook it up to source control. Ideally, I would like to make the changes in my local folder and see those changes reflected in the local cluster. With Kubernetes we can mount a volume onto the cluster and have it shared across pods.

The File System SCM plugin that will allow us to treat a folder as if it is the Source Control for our application. With that we can now mount the folder onto the cluster and Jenkins master and agents will have access to the local files.

The mounting of the folder onto the cluster is handled with the minikube mount command which will make your code folder available to the cluster. The mount command will operate from the executable window and you will need to ctrl-c it to stop the share. On Windows the mount sometimes locks up the directory if the minikube cluster is stopped before the mount command is terminated. Using minikube start --mount --mount-string on Windows has been unsuccessful and has caused OS lockups, soI would not advise using that method of mounting your code.

My jenkins-helm-values.yaml should contain all the plugins and their latest versions plus any other initialization scripts that we will use.

Now to dissect these helm values:

To begin, we specify the version of Jenkins we want to use. Then we will set up a NodePort exposed service so that we can get to Jenkins on our cluster. After that we specifying which plugins we want to install at start up of Jenkins and their versions, it is a good idea to keep this list up to date with the latest version of plugins as bugs and security holes may have been patched in later versions.

ScriptApproval section will white list any Groovy commands that you need to call in your pipeline that are outside the regular commands. This list is usually specified by the error you get when a command outside of that list is being used in the pipeline.

InitScripts is the magic that lets you configure Jenkins jobs to start without having to set them up manually. This makes it very quick for any developer to get started quickly on their local cluster. In our case, we are initializing the FileSystem SCM and setting up the initial pipeline.

Agent configuration is to specify mounting the docker daemon and the local mount path for code. Usually we can get away with the defaults but in this case we want the jenkins agent pods to have the code available.

Persistence is to specify the volume and the mount for our code.

Local Development

Now that we have our jenkins we can deploy the helm chart to our local cluster

helm install --name jenkins --values jenkins-helm-values.yaml stable/jenkins

We also need to mount our code folder as a volume to the cluster

1. Powershell: "minikube mount $($pwd.Path):/code/jobrm" | Invoke-Expression
2. CMD: minikube mount %cd%:/code/jobrm
3. Bash: minikube mount $(pwd):/code/jobrm

If you have other projects that you would like to mount, you could run multiple mounts or maybe just mount your parent directory.

To bring up jenkins run: minikube service jenkins-jenkins and minikube dashboard to bring up the kubernetes dashboard. Enter the username/password for jenkins which you can find in the secrets.

Pipeline

Now that we have everything running we need to create our pipeline for the build.

This is a very simple pipeline of 2 stages, Get Latest, and Build. The pipeline uses the angular-cli docker image to do the build of our project. Line 5 has the inheritFrom: 'default' that will use our Agent settings that we specified in our initial Jenkins configuration.

This shows how Jenkins can be our local build server, and we can now expand on the implementation of deploying the application.