The power of Helm

What is Helm?

Helm helps you manage Kubernetes applications. Helm charts help you define, install, and upgrade even the most complex Kubernetes application.

When using Kubernetes (k8s), you’ll notice that for a single deployment you’ll be managing several resources: Service, Ingress, ConfigMap and Deployment. When deploying, you may also want to do releases. These releases will want to go through different environments.

You could do this with bash scripts, but you’ll have to do all the checking and querying manually in order to ensure deployments were successful.

There might also be common services you’ll be pushing out to k8s like MongoDB and Traefik. These need a little customisation, but you wouldn’t want to maintain their k8s scripts as well.

 

Why should you care about Helm?

Deployment is a relatively complicated process, it can be full of caveats and pitfalls, nuances and intricacies. Anything to make deployment easier is a god send and welcomed by all. Helm really helps with the deployment of simple to complex k8s applications. Helm makes this process simple by giving simple commands to use, a central place to change those intricate variables and stateful management of releases and revisions (something that doesn't come with k8s by default).

 

What are we going to discuss?

The aim of this blog is to quickly highlight the power of helm, give a technically light overview of quick starting with Helm. Finally, I will discuss the problems I've hit in order to help prevent anyone else making the same mistakes.

 

So what does Helm do?

It will handle the deployment of k8s resources including versioning and releases through the creation of charts. Charts are simply configurable templates. They’re simple to make and you can take an existing k8s resource and turn hard-coded values into template variables. Helm then allows you to package up the charts and provides a full package management experience around k8s apps both local and remote.

 

Quick Look

To install Helm and Tiller (until v3 where Tiller is removed) view here 

 

What is Helm physically?

Once Helm is installed, it's just a CLI tool and some resources (Tiller) installed on your k8s cluster (up to v3).

If you run the below command from your terminal window:

$ helm version

You will be displayed something like:

Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}

Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}

You're now ready to use Helm and take advantage of its powerful framework.

 

Getting Started

When creating Helm releases you start with a Chart.yaml file - yes, all Helm configuration files are done using YAML.

apiVersion: v1 // required, can only be "v1"
name: pointsapp // required
version: 0.1.0 // required, has to be SemVar 2
appVersion: "1.0.0" // optional

This is the “entrypoint” to your Helm package.

You'll need to put this file somewhere. Helm will use the current folder name as the package name, so I have included a reference app (pointsapp) with the structure of the charts below:

pointsapp /
| chart/
| | pointsapp/
| | | ui/
| | | | templates/
| | | | |- service.yaml
| | | | |- deployment.yaml
| | | | |- ingress.yaml
| | | |- Chart.yaml // application specific Chart spec
| | | |- values.yaml // application specific Values spec
| | | server/
| | | | templates/
| | | | |- service.yaml
| | | | |- deployment.yaml
| | | | |- ingress.yaml
| | | |- Chart.yaml // application specific Chart spec
| | | |- values.yaml // application specific Values spec
| | |- Chart.yaml // package specific Chart spec <-- OUR ENTRYPOINT 
| | |- values.yaml // package specific Values spec
| ui/
| | // react frontend
| |- index.js
| |- package.json
| server/
| | // nodejs backend
| |- index.js
| |- package.json
|- docker-compose.yml
|- README.md

 

As you can see, I’ve created a chart directory from this structure, which will house my Helm package. Inside the chart directory, I’ve created a pointsapp directory. This will be the name of the package and will contain all Helm charts and templates. It's also where that initial Chart.yaml goes, as I commented in the directory structure, it's the whole package specific chart spec. (I call it a chart spec - it's just the chart definition really, but sounds cooler using the word "spec")

You may have also noticed that there are templates directories. These house your k8s resource configurations.

 

Templates

What are templates? They are k8s resources - with a twist. The twist being that you can interpolate values (swapping out defined bits for the real values) from other files into them. You may have noticed the values.yaml files. This is one type of file from which the values are taken and interpolated into the templates.

So, before going into the types of interpolation that can be done, or where they come from, it's important to understand what this actually means, and how it's done.

Below is a resource file:

apiVersion: v1
kind: Service
metadata:
  name: --service
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort:
  selector:
    app: -
status:
  loadBalancer: {}

Hopefully, you’ll see these weird blocks and to name but a few. These are defined bits ready to be interpolated.

The first thing to understand about these expressions is that they’re from the Go Templating language.

The braces mean 'start interpolating here'. The dots are akin to accessing keys in objects like you do with OOP languages (inc JavaScript). The JavaScript block below demonstrates this:

var myObj = { Values: { service: { name: 'TheService' } } }
console.log(myObj.Values.service.name)
// logs out TheService

It's important to remember that the myObj is JSON (JavaScript Object Notation) and that YAML is a superset of JSON. So, the YAML configuration creates objects and arrays, whilst the Go Templating language allows you to access these objects in a similar fashion to accessing JSON in JavaScript.

The initial dot means the current context (context of Helm). The first key is actually the file you wish to read from. Helm gives you several including values, chart and release. Chart and release are Helm specific, therefore have particular conventions for accessing the resources.

What's more interesting is the values key. You can define your own values.yaml to inject values into your templates:

global:
  env: production
service:
  authKey: someAuthKey
  mongoUrl: "mongodb+srv://main:[SomePassord]@[SomeCluster]/PointsApp?retryWrites=true&w=majority"
  containerPort: 80
ui:
  apiHost: http://points-api
  containerPort: 443

This is just plain old YAML. We define our objects with key pair values. This means that now all of the resources get access to a common variable set rather than having to do a copy paste job manually into all our k8s resources.

The output after running this command:

$ helm template pointsapp > sample.yaml

Is:

---
# Source: pointsapp/charts/pointsapp-api/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: pointsapp-api-release-name-service
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  selector:
    app: pointsapp-api-release-name
status:
  loadBalancer: {}

Here you can see that the values have been interpolated from multiple sources including Helm itself, and a values.yaml.

 

What's the power of this?

Hopefully you can now start to see the power of Helm. K8s allows you to declaratively describe your applications with target states and container deployments. With a Helm package, you can now group your applications in one package and specify “global” values to be interpolated into them, handling the orthogonal concerns of releases and deployments. This means you can specify multiple target states depending on whatever needs to be relied on.

This makes continuous delivery (CD) a lot easier. At the end of the build process, rather than creating artefacts you can create two things: Docker Images and Helm Packages. These can then be deployed in k8s via one command:

$ helm install pointsapp

What else do you get with this?

There are more benefits! (This is in no way an exhaustive list)

 

Releases and revisions

Helm packages are versioned and when released, are given a release name (automatically generated or manually assigned). 

$ helm list
NAME         REVISION  UPDATED            STATUS    CHART            APP    VERSION  NAMESPACE
Crazy-horse  1         Fri Sep ... 2019   DEPLOYED  pointsapp-0.1.0  1.0.0           default

This displays the status, release name, revision and application that was deployed.

You may have noticed in the templates there is used as the app metadata label. This means you can do multiple releases of the same application into the same namespace in k8s - it's pretty awesome. Why? Because the app name can now be dynamically changed with all selectors being replaced with the same names. So the app in k8s is actually "crazy-horse-pointsapp" which to k8s is totally different to "flying-bear-pointsapp".

Template validation can be done with: 

$ helm template pointsapp

As discussed before, if you want further validation you can take that output (above described piping into a file) and pass it to k8s:

$ kubectl apply -f sample.yml --server-dry-run

This is the equivalent of having a template validation step. If your build server has k8s installed onto it, it means during your build process, not only can you have Helm validation, you can also have k8s validation. You have now super enhanced our CD.

Copying YAML blocks directly into templates allows you to write configuration blocks directly into values.yaml and have it verbatim copied into any template.

Sample Chart creation is as simple as:

$ helm create mynewchart

This will give you a really comprehensive sample of most things you can do with Helm templates. 

 

Kubernetes VS Code Extensions

This amazing little tool not only shows you all of your k8s clusters and resources (much like the dashboard does), but also provides Helm syntax, highlighting and IntelliSense. In addition to this, it shows local and remote Helm packages.

 

Sticking Points

All this being said, there have been areas which have held me back, somewhat. They’re not necessarily problems with Helm, more of deep dark holes of despair that I’ve accidentally fallen into. You can find these below:

Helm Ignore File:

This file had it in for me! Before I knew about this file and created my amazing chart directory structure inside my application - which was not contained within a chart directory, so it sat along side all of project files - every time I ran install, the below error came up:

Error: grpc: received message larger than max

After looking through several GitHub issues and threads someone mentioned that if I had a Node modules folder in there then it was trying to package that up.

First caveat: running Helm install will try and gather all the files and traverse all the directories from where you execute it.

Solution: use the Helm ignore file.

The first part of solving this was to create this ignore file, and to group all charts and templates into the structure I outlined in 'getting started'.

The next problem I encountered is that my server code directory is called “server”. My server chart directory is also called "server" and I blindly added "server" to the helm ignore directory. D’oh…

The error I got from this was:

Error: release harping-ant failed: no objects visited

The bit to focus on is "no objects visited", this is because it found no template YAML files.

Solution: This is twofold. Firstly, I made the directory structure as outlined in the 'getting started' section, and moved my Helm ignore file in there. This means there was less to ignore (just DS_Store at this point) and it was easier to traverse. The next step was to remove the ignore directories which now works properly, because when you run Helm install, it won’t encounter any of your source code files.

Integers in K8s Deployment container spec environment variables:

Some may not know that when you create a containers env object, you can’t put numbers into it as per the k8s API docs

However, other places in your k8s resources may want to use the int representation, especially if you’re referring to network port numbers.

My issue was here:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: --deployment
  labels:
    app: -
spec:
  replicas: 1
  strategy: {}
  selector:
    matchLabels:
      app: -
  template:
    metadata:
      labels:
        app: -
    spec:
      containers:
      - env:
        - name: AUTH_KEY
          value:
        - name: DATABASE_URL
          value:
        - name: ENVIRONMENT
          value:
        - name: HOST
          value: "0.0.0.0"
        - name: PORT
          value: // <-- NOT ALLOWED
        - name: MONGO_NEW_URL_PARSER
          value: "true"
        image: dockerhub/dev:api
        name: -
        ports:
        - containerPort:
        resources: {}
      restartPolicy: Always
      imagePullSecrets:
      - name: docker-reg
status: {}

Look at the "spec: containers: -env" list, and see the key "- name: PORT", just below I have the incorrect usage of my variable. This variable is configured as an integer in the Values file, but in the Env, it needs to be a string.

The answer doesn’t lie within the Values.yaml. It's far more simple than that.

Solution: Wrap the with double quotes. Because this is interpolating values, there's no need to start converting ints to strings with Helm template functions, or add another variable to the values.yaml. You just make sure the outputted YAML looks like a string - simple. 

The solution in templates:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: --deployment
  labels:
    app: -
spec:
  replicas: 1
  strategy: {}
  selector:
    matchLabels:
      app: -
  template:
    metadata:
      labels:
        app: -
    spec:
      containers:
      - env:
        - name: AUTH_KEY
          value:
        - name: DATABASE_URL
          value:
        - name: ENVIRONMENT
          value:
        - name: HOST
          value: "0.0.0.0"
        - name: PORT
          value: "" // <-- SOLVED!
        - name: MONGO_NEW_URL_PARSER
          value: "true"
        image: dockerhub/dev:api
        name: -
        ports:
        - containerPort:
        resources: {}
      restartPolicy: Always
      imagePullSecrets:
      - name: docker-reg
status: {}

 

Values Values Values:

So the first thing about values.yaml is, if you use make sure you create a values.yaml. You can have child charts, but they can’t see the parents. Parents override children. In directory structure initially I had this:

pointsapp /
| chart/
| | pointsapp/
| | | ui/
| | | | templates/
| | | | |- service.yaml
| | | | |- deployment.yaml
| | | | |- ingress.yaml
| | | |- Chart.yaml // application specific Chart spec
| | | server/
| | | | templates/
| | | | |- service.yaml
| | | | |- deployment.yaml
| | | | |- ingress.yaml
| | | |- Chart.yaml // application specific Chart spec
| | |- Chart.yaml // package specific Chart spec
| | |- values.yaml // package specific Values spec

As you can see, I have a "root" values.yaml file in the chart/pointsapp directory. I have no "application" specific value files. This will make Helm show an error stating that it can’t find the keys in the values.

Solutions, there are two:

First, make "application" specific values.yaml and copy just the "application" specific values across in the exact same structure.

The second solution is to remove the "package" specific values.yaml in favour of having only "application" specific - that way the structure of the values can be flatter.

After you apply the first solution - as discussed at the beginning, parent overrides child, so "package" specific values.yaml overrides "application" specific values.yaml.

 

Parallels

Sometimes a way to get information to seep into the mind quicker than usual is to draw parallels between concepts, because this way you’re not starting from scratch.

There are 3 things to think of:

First, it allows more complex templating - so is simply a templater.

Secondly, it's a package manager.

Finally, if you think that Docker has Dockerfile and to coordinate multiple Docker related services you have Docker Compose, allowing you have multiple compose files mashed into one - it's simply Docker Compose for K8s.

For me, I simply think of it as a Package Manager and Docker Compose for K8s. But it's good to remember that Helm is an extremely powerful tool, and the more imagination you have the crazier things you can get it to do.

 

Conclusion

With great power etc. etc. yada yada yada… Please go out and have fun with this!

This is an incredibly useful tool and lets you be very versatile and quick with building a Kubernetes environment. I got started in minutes with Grafana, Traefik and deploying my app. Beware though, running all this on a local laptop will drain all the battery :).