How to Install Rancher Labs k3s on Raspbian or Ubuntu

Add Cgroups to Linux

Before jumping into Kubernetes, Linux needs to be setup for virtualization.

Tip: Read how I made the switch from 32-bit Raspbian to 64-bit Ubuntu to improve Docker image compatibility. Jump to Ubuntu Post.

Containers and virtual machines rely on Linux namespaces and control groups for security, so we need to enable them in Raspbian by editing the /boot/cmdline.txt file. Reboot the nodes when you finish.

Enable Virtualization for Raspbian

sudo cp /boot/cmdline.txt /boot/cmdline.txt.bak1
orig="$(head -n1 /boot/cmdline.txt) cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory"
echo $orig | sudo tee /boot/cmdline.txt
sudo reboot

Enable Virtualization for Ubuntu

sudo cp /boot/firmware/cmdline.txt /boot/firmware/cmdline.txt.bak1
orig="$(head -n1 /boot/firmware/cmdline.txt) cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory"
echo $orig | sudo tee /boot/firmware/cmdline.txt
sudo reboot

Warning: If you forget this step, the k3s agent won’t work. I build all my nodes manually, and I have accidentally skipped this step. When I do, the k3s agent appears to install correctly, but it never shows up with “kubectl get nodes.” Digging into the logs you find that the agent won’t start without the Linux CPU and memory control groups.

Installing the Master Node

It is straightforward to get started with K3s. See the Quick Start Guide. I installed k3s slightly differently using the command below. It downloads an installation script, pipes it into the shell to install and configure the master node, and then starts the server. The Kubeconfig config is stored in /etc/rancher/k3s/k3s.yaml during the process.

Warning: If running Ubuntu, replace “ssh pi@p…” with “ubuntu@pi…” or add the user “pi” to Ubuntu with sudo privileges

curl -sfL https://get.k3s.io | sh -s - server &

When the server starts, verify the installation with this command:

$ sudo k3s kubectl get nodes
NAME   STATUS   ROLES    AGE     VERSION
pi1    Ready    master   3m15s   v1.19.3+k3s3

Next, copy the kubeconfig file and change the server IP address from 127.0.0.1 to the master node IP address. You will use this copy of the kubeconfig file to access k3s from your computer.

sudo cp /etc/rancher/k3s/k3s.yaml ~/kubeconfig
sudo chown $(whoami):$(whoami) ~/kubeconfig

Copy the edited file to your computer. Export the KUBECONFIG variable from the shell or export it from your ~/.zshrc or ~/.bashrc file.

scp pi@pi1:~/kubeconfig ~/kubeconfig
export KUBECONFIG=~/kubeconfig

Verify that your laptop or desktop computer is configured correctly by getting a list of the nodes:

$ kubectl get nodes
NAME   STATUS   ROLES    AGE     VERSION
pi1    Ready    master   3m15s   v1.19.3+k3s3

Installing the Agent Nodes

List the server token from the master node, copy it, and save it for later.

ssh pi@pi1
sudo cat /var/lib/rancher/k3s/server/node-token

Pass the server token into the agent node command to install the k3s agent on the remaining Raspberry Pis in your cluster. You can do this on the nodes individually or use the tip below to install the agent on all of the nodes simultaneously.

Tip: Would you like to be able to update all of Raspberry Pis at simultaneously? If so, see my side-note How to Multicast Commands.

Whether you update the nodes one at a time or simultaneously, export the server token copied above as a variable, and then run the command shown. When I installed Ubuntu, I broadcast the commands to all the worker nodes at the same time and it worked great.

ssh pi2
export TOKEN=***REDACTED***
curl -sfL https://get.k3s.io agent | INSTALL_K3S_EXEC="agent --server https://192.168.1.10:6443 --token $TOKEN" sh -

Wait a couple of minutes while the agent starts-up and registers with the main node. Re-run “kubectl get nodes.” You should see results like those shown below.