My DevOps Homelab 🧪

Wednesday, May 26, 2021

Why would I want to run kubernetes at home?

github.com/brittonhayes/devops-lab

A few reasons

api: v1
metadata:
  name: kubernetes-is-hard
  namespace: kube-system
  labels:
    details: "Becoming comfortable with it under pressure is even harder"

Setting up a DevSecOps Homelab with a really simple kubernetes setup allows me to tackle this in a few ways:

  1. I need a way to test out ideas in a safe environment that doesn’t affect my production or staging environments at work.
  2. Kubernetes is a strange beast, so making it a part of my home setup makes it all that much more familiar.
  3. Flux and K3OS make it so easy to setup it’s mind-blowing; so why not?

Kubernetes is a pretty substantial part of my work environment, but the enterprise scale version of anything can be a very daunting place to get familiarized with new technologies or concepts. Thus, I opted into the wonderful world of Raspberry Pi, NUCs, and k3OS.

What is k3OS?

“k3OS is purpose-built to simplify Kubernetes operations in low-resource computing environments. Installs fast. Boots faster. Managed through Kubernetes.”

I have k3OS installed across each of my Raspberry Pi 4s as well as my Intel NUC. Any new node I want to add onto the cluster is as easy as a flashing an SD card and running something like the following.

export AGENT_IP=192.168.0.101
export SERVER_IP=192.168.0.100

k3sup join --ip $AGENT_IP --server-ip $SERVER_IP

This command instantly joins any new node you’ve added into the cluster with no other work required. Pretty cool, right?

Managing Deployments with Flux CD

“Flux is a collection of tools for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.”

Flux is wonderful. I can essentially just define all of my services, deploys, ingresses etc. as YAML manifests and flux will regularly check my Github repository for changes so Git is always the source of truth. It keeps things very simple and clearly defined in code.

If you’re new to this concept, it’s commonly referred to as GitOps. It’s a way of managing your infrastructure and applications so that whole system is described declaratively and version controlled.

So what’s it look like?

Here is a copy-pasted example of my home assistant helm release. Every 15 minutes flux will go check to make sure there have been no changes to my chart, values, or helm repository. There is no need for kubectl apply or waiting for a Spinnaker deploy; this just works. Write the yaml and all done.

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: home-assistant
  namespace: home
spec:
  releaseName: home-assistant
  targetNamespace: home
  interval: 15m
  chart:
    spec:
      chart: home-assistant
      version: v8.2.0
      sourceRef:
        kind: HelmRepository
        name: k8s-at-home
        namespace: flux-system
      interval: 15m
  values:
    service:
      type: LoadBalancer
      externalTrafficPolicy: Local
    persistence:
      config:
        enabled: true
    ingress:
      enabled: true
      ingressClassName: "traefik"
      annotations:
        traefik.ingress.kubernetes.io/router.entrypoints: "websecure"
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: home-assistant
  namespace: kube-system
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: Host(`home-assistant.lan`)
      services:
        - name: home-assistant
          port: 8123
  tls:
    certResolver: default
    options: {}

To make some of the applications easier to access and require a little less setup with an IngressRoute, I use Metallb. MetalLB is a load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols. I have a Traefik ingressroute here too for demonstration purposes but sometimes you just wanna directly access a baremetal IP and Metallb keeps it simple for the homelab.

Wrapping Up

This is a just a brief little look at how I’ve started to run kubernetes at home. So far the process has been a blast and after doing this for a while now, I feel substantially more comfortable and familiar with the quirks of working with k8s. If you’d like to see more examples of how the homelab works, feel free to glance around at my repository!

github.com/brittonhayes/devops-lab

Thanks for reading and see ya next time!