Kubernetes (mikrok8s)

Prefer ubuntu based host for mikrok8s, you can use lima-vm or wsl2 distro. you can follow the instructions here for installing mikrok8s.

kubectl config

Make sure ~/.kube exists

mkdir -p ~/.kube

Create the config file

microk8s config > ~/.kube/config

Execute cluster-info to verify the connection

kubectl cluster-info

Microk8s addon

Enable host-access addon for convenient way to access the host from inside the cluster.

microk8s enable host-access

Since tilt need a local registry to push the images, you can enable the local registry addon.

microk8s enable registry

later if you want to free some spaces from the registry, you can run the following command:

microk8s disable registry
microk8s disable hostpath-storage:destroy-storage
microk8s enable registry

Production Lessons & Troubleshooting

AppArmor / Container-Env Failure After Reboot

On certain Ubuntu installations or environments where systemd-detect-virt reports a container (for instance, if /.dockerenv is present), systemd skips mounting securityfs at boot. Consequently:

apparmor.service fails its assertion.
snapd.apparmor.service skips loading snap AppArmor profiles with Inside container environment without internal policy.
snap.microk8s.daemon-containerd and other MicroK8s daemons crashloop with missing profile snap.microk8s.microk8s or aa_is_enabled() failed unexpectedly.

Manual Recovery Steps:

If MicroK8s fails to start after a reboot due to this issue:

# 1. Mount securityfs
sudo mount -t securityfs securityfs /sys/kernel/security

# 2. Start AppArmor and parse snap profiles
sudo systemctl start apparmor.service
sudo apparmor_parser -r -W /var/lib/snapd/apparmor/profiles/

# 3. Reset failed state and restart MicroK8s daemons
sudo systemctl reset-failed snap.microk8s.daemon-*
sudo systemctl start snap.microk8s.daemon-containerd \
                     snap.microk8s.daemon-k8s-dqlite \
                     snap.microk8s.daemon-kubelite \
                     snap.microk8s.daemon-cluster-agent

Permanent Fix:

Use a systemd oneshot unit or an Ansible playbook to ensure securityfs is mounted and snap AppArmor profiles are loaded at boot prior to starting snap.microk8s.daemon-*.service.

Storage & Ingress Realities

Storage: While hostpath-storage is fine for simple local testing, production workloads needing persistence on dedicated directories (or multi-node portability) should use carefully scoped HostPath volumes, local path provisioners, or external replication (such as Litestream for SQLite).
Ingress: In practice, pair MicroK8s with Traefik or an Ingress/Gateway API controller and MetalLB for predictable IP allocation on your local network or Tailscale mesh.

homelabguide