Deploying Kubernetes on Ubuntu Bare Metal Micro-Servers To Build A Home Lab

I recently decided to set up a homelab with cheap micro-servers. To ease the networking setup, so that I don’t have to configure static IP addresses for each machine, I used Tailscale—it gives each node a stable 100.x.y.z address without DHCP headaches. I wanted the environment to be Kubernetes (I know, overkill)—but it’s what I use at work and is an industry standard, so it’s nice to learn more about the infra I operate in. This blog is just a short guide on getting this running and some of the challenges I faced along the way. I also learned a bit about Kubernetes RBAC when trying to make a worker node join the cluster. Here it goes!

1. Micro-Server Preparation Micro-Server Preparation

Preparation steps for all nodes (control‑plane and worker):

# Disable swap for Kubernetes
sudo swapoff -a
# Enable bridged traffic through iptables
sudo modprobe br_netfilter
sudo sysctl -w net.bridge.bridge-nf-call-iptables=1

Disabling swap prevents the kube-scheduler from being misled by swapped-out memory, ensuring accurate resource accounting. Loading br_netfilter and enabling net.bridge.bridge-nf-call-iptables makes sure that bridged network traffic (used by most CNI plugins) is correctly forwarded through your host’s iptables rules.

Installation of containerd, kubeadm, kubelet, and kubectl via official repositories ensures compatibility. On Ubuntu 24, execute:

# Install containerd
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
  | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y containerd.io
sudo systemctl enable --now containerd

# Install Kubernetes tools
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-archive-keyring.gpg
echo \
  "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] \
  https://apt.kubernetes.io/ kubernetes-xenial main" \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubeadm kubelet kubectl
sudo apt-mark hold kubeadm kubelet kubectl

Each node then joins the private Tailscale mesh to receive a stable 100.x.y.z address. Each node joins the private Tailscale mesh to receive a stable 100.x.y.z address.

2. Initialize the Control-Plane on Tailscale IP

TSIP=$(tailscale ip -4)  # e.g. 100.106.211.69

sudo kubeadm init \
  --apiserver-advertise-address=${TSIP} \
  --control-plane-endpoint=${TSIP}:6443 \
  --pod-network-cidr=192.168.0.0/16

# Make kubectl work for your user
mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

This generates the signed cluster-info ConfigMap and core RBAC, but we still need to tweak permissions for join.

3. Essential RBAC Tweaks

3.1 Anonymous Discovery of `cluster-info`

Issue: Bootstrap token discovery kept failing with system:anonymous cannot get resource "configmaps" even though cluster-info existed. Resolved via https://github.com/kubernetes-sigs/kubespray/issues/4117 and https://github.com/kubernetes/website/pull/19868/files.

# Role in kube-public
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: kube-public
  name: kubeadm:bootstrap-signer-clusterinfo
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["cluster-info"]
  verbs: ["get"]


# Let system:anonymous read it
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: kube-public
  name: kubeadm:bootstrap-signer-clusterinfo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubeadm:bootstrap-signer-clusterinfo
subjects:
- kind: User
  name: system:anonymous

Low Risk: The ConfigMap contains only the API endpoint, CA cert, and a JWS signature—no secrets. Anonymous GET is read-only and tamper-proof. The ConfigMap contains only the API endpoint, CA cert, and a JWS signature—no secrets. Anonymous GET is read-only and tamper-proof.

3.2 Bootstrap Token Permissions

Issue: After anonymous discovery, kubeadm join failed on component-config fetch: configmaps "kubelet-config" is forbidden. Fixed by creating a ClusterRoleBinding for the bootstrapper group (see https://github.com/kubernetes/website/pull/19868/files).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubeadm-bootstrap-node-perms
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: Group
  name: system:bootstrap:1dy4hs         # from `kubeadm token list`
  apiGroup: rbac.authorization.k8s.io

This grants the token rights to get Nodes and component-config ConfigMaps (kubeadm-config, kubelet-config, kube-proxy). (kubeadm-config, kubelet-config, kube-proxy).

4. Upload Component-Config Manifests

Run on the control-plane to push down matching configs for your kubeadm version:

for phase in kubeadm kubelet kube-proxy; do
  sudo kubeadm init phase upload-config ${phase} \
    --kubeconfig /etc/kubernetes/admin.conf || true
done

5. Join Your Workers

On each worker, simply run:

TSIP=100.106.211.69
sudo kubeadm join ${TSIP}:6443 \
  --token 1dy4hs.uv9soyrbhwvo4g4d \
  --discovery-token-ca-cert-hash sha256:<your-hash>

Watch it succeed:

[discovery] Successfully fetched signed cluster-info ConfigMap
[preflight] Downloading component configs
[join] This node has joined the cluster

6. Schedule Pods on the Master

By default the control-plane is tainted to avoid user workloads. To allow pods on your micro-server master:

kubectl taint nodes $(hostname) node-role.kubernetes.io/control-plane-

Now you can deploy pods (e.g. your Dashboard or small cronjobs) on that same box.

7. Automation Snippets

A) Bash Control-Plane Bootstrap

#!/usr/bin/env bash
set -euo pipefail

TSIP=$(tailscale ip -4)
sudo swapoff -a
sudo modprobe br_netfilter
sudo sysctl -w net.bridge.bridge-nf-call-iptables=1

sudo kubeadm init \
  --apiserver-advertise-address=${TSIP} \
  --control-plane-endpoint=${TSIP}:6443 \
  --pod-network-cidr=192.168.0.0/16

mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Apply RBAC definitions
kubectl apply -f rbac-bootstrap-clusterinfo.yaml
kubectl apply -f rbac-bootstrap-node-perms.yaml

# Push component configs
for phase in kubeadm kubelet kube-proxy; do
  sudo kubeadm init phase upload-config $phase --kubeconfig /etc/kubernetes/admin.conf || true
done

echo "Control-plane ready—use 'kubeadm token create --print-join-command' for workers."

B) Ansible Task List

- hosts: control-plane
  become: yes
  tasks:
    - name: Prep system
      shell: |
        swapoff -a
        modprobe br_netfilter
        sysctl -w net.bridge.bridge-nf-call-iptables=1

    - name: kubeadm init control-plane
      command: >
        kubeadm init
          --apiserver-advertise-address={{ tailscale_ip }}
          --control-plane-endpoint={{ tailscale_ip }}:6443
          --pod-network-cidr=192.168.0.0/16

    - name: Copy kubeconfig for user
      copy:
        src: /etc/kubernetes/admin.conf
        dest: "{{ ansible_env.HOME }}/.kube/config"
        owner: "{{ ansible_user_id }}"
        mode: 0600

    - name: Apply anonymous bootstrap RBAC
      k8s:
        definition: "{{ lookup('file','rbac-bootstrap-clusterinfo.yaml') }}"
      environment:
        KUBECONFIG: /etc/kubernetes/admin.conf

    - name: Bind bootstrap token to view
      k8s:
        definition: "{{ lookup('file','rbac-bootstrap-node-perms.yaml') }}"
      environment:
        KUBECONFIG: /etc/kubernetes/admin.conf

    - name: Upload component configs
      shell: |
        for phase in kubeadm kubelet kube-proxy; do
          kubeadm init phase upload-config $phase --kubeconfig /etc/kubernetes/admin.conf || true
        done

Hopefully this helps someone who learning the fundamentals of k8s.