While managing my kubernetes cluster, I needed to install several helm charts to do things like:

• Installing ingress controller in order to connect my cluster's ingress with my cloud-provider's load balancer
• Installing metrics-server to be able to get basic metrics about my cluster

In doing so, I found myself writing commands like

helm repo add bitnami https://charts.bitnami.com/bitnami

helm upgrade --install metrics-server bitnami/metrics-server \
  --create-namespace --namespace metrics-server \
  --set apiService.create=true \
  --set extraArgs.kubelet-insecure-tls=true \
  --set extraArgs.kubelet-preferred-address-types=InternalIP \
  --set hostNetwork=true

in the command line to install the required resources then forget about it.

I didn't like this, as it does not document what I have installed, and does not document how I installed it, and if I needed to install the same infrastructure on another cluster (say from another provider) I will need to scan through my command line history to find the commands I used to install the needed resources.

I began experimenting with creating my own helm chart for my setup, specifying all the required dependencies and so. But as I learned more about it I felt it was too much for what I wanted.

Enters helmfile

Searching about a simpler solution, I found helmfile. The whole idea is that you create a yaml file consisting of your dependencies charts, and you use helmfile to install them.

It was exactly what I wanted.

The configuration file

First, we need to write a yaml file that describes the helm charts that needs to be installed, and specify some details about the related repo, namespace, and other things.

Let's take another look at the commands I ran earlier

helm repo add bitnami https://charts.bitnami.com/bitnami

helm upgrade --install metrics-server bitnami/metrics-server \
  --create-namespace --namespace metrics-server \
  --set apiService.create=true \
  --set extraArgs.kubelet-insecure-tls=true \
  --set extraArgs.kubelet-preferred-address-types=InternalIP \
  --set hostNetwork=true

We can extract some values from those 2 commands, like repo URL (charts.bitnami.com/bitnami), chart name (bitnami/metrics-server), release name (metrics-server), installation namespace (metrics-server), and some flags (--create-namespace flag), and some values that we set using --set flag.

From the above information, and with some help from the documentation of helmfile we can create the following schema

repositories:
  - name: bitnami
    url: https://charts.bitnami.com/bitnami

releases:
  - name: metrics-server
    chart: bitnami/metrics-server
    namespace: metrics-server
    createNamespace: true
    set:
      - name: apiService.create
        value: true
      - name: extraArgs.kubelet-insecure-tls
        value: true
      - name: extraArgs.kubelet-preferred-address-types
        value: InternalIP
      - name: hostNetwork
        value: true

Now we are only missing 1 required field for the release, and that is the version of the chart. We can get the version number by running

helm repo add bitnami https://charts.bitnami.com/bitnami
helm show chart bitnami/metrics-server

I.e. we add the repo, then call helm show chart {repo}/{chart}.

As of the time of writing this article, the latest version of the metrics-server is 6.0.7, but it has a bug so I am using an older version of 5.11.3

Digging through the documentation of helmfile, we can also make use of atomic option that restores previous state in case of failed release, so that the cluster does not change if the installation failed.

On last thing that I like to do is to set the cluster context in the configuration file, I manage several clusters, and I don't want one cluster's configurations to be applied onto another one, so in the root of the configuration file I added:

helmDefaults:
  kubeContext: my-cluster-context

Now, the final configuration file looks like:

helmDefaults:
  kubeContext: my-cluster-context

repositories:
  - name: bitnami
    url: https://charts.bitnami.com/bitnami

releases:
  - name: metrics-server
    chart: bitnami/metrics-server
    version: 5.11.3
    namespace: metrics-server
    createNamespace: true
    atomic: true
    set:
      - name: apiService.create
        value: true
      - name: extraArgs.kubelet-insecure-tls
        value: true
      - name: extraArgs.kubelet-preferred-address-types
        value: InternalIP
      - name: hostNetwork
        value: true

Applying The Configurations

We could download the binaries from Github repo, then install some dependencies, then use that binary to install our configuration, but that's not my favorite method to do it.

My favorite method is by using Docker.

In their installation section they have a section for "Running as a container", so let's use that.

I created a small bash file based on the command in the documentation, and I put it in the same directory as the configuration file we made earlier. Here are the contents of the bash file...

# This script deploys the helmfile using helmfile docker image.

# Save current context
current_context=$(kubectl config current-context)

# Write config to ~/.kube/config
kubectl config view --flatten > ~/.kube/config

# Use the saved context
kubectl config use-context $current_context

docker run \
--rm \
--net=host \
-v "${HOME}/.kube:/root/.kube" \
-v "${HOME}/.config/helm:/root/.config/helm" \
-v "${PWD}:/wd" --workdir /wd \
quay.io/roboll/helmfile:helm3-v0.135.0 helmfile sync

The command kubectl config view --flatten > ~/.kube/config is used to write all my kubernetes contexts into the file ~/.kube/config as the docker image reads the contexts from that file.

The first and the third commands are to keep the state of the current context, as it tends to change when I write the contexts into the config file.

All what's left is to run the bash file. It will take a minute, then print something like the following...

Adding repo bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

Affected releases are:
metrics-server (bitnami/metrics-server) UPDATED

Upgrading release=metrics-server, chart=bitnami/metrics-server
Listing releases matching ^metrics-server$
Release "metrics-server" has been upgraded. Happy Helming!
NAME: metrics-server
LAST DEPLOYED: Sat Jun 18 04:38:44 2022
NAMESPACE: metrics-server
STATUS: deployed
REVISION: 5
TEST SUITE: None
NOTES:
CHART NAME: metrics-server
CHART VERSION: 5.11.3
APP VERSION: 0.6.1

** Please be patient while the chart is being deployed **

The metric server has been deployed.

In a few minutes you should be able to list metrics using the following
command:

kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes"

metrics-server  metrics-server  5               2022-06-18 04:38:44.662123115 +0000 UTC deployed        metrics-server-5.11.3   0.6.1

And that's it, metrics server should now be installed in a namespace called "metrics-server" and we should be able to get some basic metrics about the cluster by running kubectl top nodes.

We can now maintain that configuration file to include any dependency that we need to install using helm. The configurations can, and should be, pushed to version control; we can also setup some simple CI/CD to auto deploy the file on any changes.