Installing a multi-node Kubernetes cluster with Ubuntu Multipass

There are tons of tutorials out there for installing Kubernetes. I am just documenting my homelab installation. I like to install Kubernetes without external tools, but relying on kubeadm and using actual Linux installations in VMs, it gives me more control and helps me to understand how it actually works, compared to alternatives like minikube and kind.

First, I will create 3 VMs with Ubuntu Multipass. The reason I have chosen it is that it makes it really easy to create virtual machines on demand. It’s not rocket science, it works with QEMU under the hood, but it makes the management of it easy by configuring the network bridging, disk images and non-ending list of QEMU variables. It has a potential to become the Docker of creating virtual machines, but right now only Ubuntu images are available. As an alternative, one can choose Weave Ingite, which is based on Firecracker MicroVM, but boots compatible kernels with OCI images under the hood. It’s an interesting project to checkout, I will be working on it soon.

The following scripts creates the VMs. If the disk images are available, it takes approximately 30 seconds per VM to be available.

$ multipass launch -c 4 -d 50G -m 8G -n node1
$ multipass launch -c 4 -d 50G -m 8G -n node2
$ multipass launch -c 4 -d 50G -m 8G -n node3

# Verify all worked correctly.
$ multipass info --all
Name:           node1
State:          Running
Release:        Ubuntu 18.04.4 LTS
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           0.08 0.02 0.01
Disk usage:     1006.5M out of 48.3G
Memory usage:   99.5M out of 7.8G

Name:           node2
State:          Running
Release:        Ubuntu 18.04.4 LTS
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           0.04 0.03 0.01
Disk usage:     1006.1M out of 48.3G
Memory usage:   102.0M out of 7.8G

Name:           node3
State:          Running
Release:        Ubuntu 18.04.4 LTS
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           0.09 0.04 0.01
Disk usage:     1007.7M out of 48.3G
Memory usage:   103.2M out of 7.8G

At the time of wrtiting this blog, I’m using Ubuntu 20.04 Beta, but multipass defaults to 18.04 for stability reasons. Following the official documentation of installing Kubernetes, I have selected Docker for container runtime, the installation can be found here.

After ensuring docker, kubeadm, kubectl is installed in each node, we can go ahead with actually creating the cluster. One caveat though; in order to create proper multi-node networking, we need to select a network add-on before creating the cluster. It’s written in the docs, but can be easily overlooked. I have chosen Calico as it’s the easiest one. Also my home network is on and to ensure it does not conflict with my home network and the network multipass builds, I will create a network in the I probably won’t need 2^20 IP, but anways.

# Kick start the cluster creation in node1
$ kubeadm init --pod-network-cidr=
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join --token h6r5uv.3vwcmbxtlkzwpwow \
  --discovery-token-ca-cert-hash sha256:bc23f307454945755ffe5ed542e2732d79baedcbefc95a92f856d13456333265 

After setting up kubectl, you can join the node2 and node3 to the cluster. run the kubeadm join command as specified in the output of the kubeadm init in the first node. The nodes will pull the related Docker images and kubelets will join the cluster really quick.

$ kubectl get node
node1   NotReady   master   5m42s   v1.18.1
node2   NotReady   <none>   5m15s   v1.18.1
node3   NotReady   <none>   5m1s    v1.18.1

You see the nodes are there, but they are not ready. Because the network add-on is not setup, and they cannot communicate properly. Download the manifest of Calico network and remember to change the CIDR and apply it.

$ wget
$ vi calico.yaml # Chagne to
$ kubectl apply -f calico.yaml
# Wait a little
$ kubectl get node
node1   Ready    master   8m50s   v1.18.1
node2   Ready    <none>   8m23s   v1.18.1
node3   Ready    <none>   8m9s    v1.18.1

As you see the installation was really quick! However, I would like the opportunity to manage my cluster in the host system, not requiring to use multipass to shell into the instance. To do that, we need to copy the kubeconfig file.

$ mkdir ~/.kube
$ multipass copy-files node1:.kube/config ~/.kube/
$ kubectl get node

Yeah, it’s working! It’s a pleasant experience. Most people think installing Kubernetes is hard, but it is actually not. For simpler use cases, Minikube works great. Docker for Desktop is also good enough experience for macOS and Windows. For my own experiments, I need raw Kubernetes installation, without any hacks and full access to the kubelet hosts. Kubeadm works very fast and requires minimal configuration. Of course, running a production ready Kubernetes cluster is a complicated task and you are probably best using managed Kubernetes services. I did install on my machine because I need it for Ph.D experiments, and I have many cores and memory to spare and it is much cheaper.

Finally, I want to give a shout out to Ubuntu for developing multipass. It also works on macOS and Windows. It’s faster than similar tools. You can close the VMs with multipass stop --all and start them again with multipass start --all. The nodes will get the same IP address (need to confirm it’s always the case) and you can resume working since kubelet and the etcd will be started.


comments powered by Disqus