Since a long time I wanted to create my own blog to brain dump all my stuff I am working on. After some rearrangements I found two Raspberry PIs which were not in use. Therefore I decided to create a mini Kubernetes cluster on them and create this blog. Due to the fact IPv4 is not available on non business internet contracts I need to choose IPv6.
Hardware Setup
To run this Blog I am using two Raspberry PI 4 devices. One with 2GB RAM and the other device with 4GB of RAM.
Software Setup
The used Software for this blog is the Operating System Ubuntu 22.04 LTS which has been installed on SD Cards with Raspberry Pi Imager
Kubernetes will be installed with K3S.
The software Ghost will be installed via a Helm chart provided by bitnami.
Installing the Operating System
The installation can be customised before flashing the Image to the disk. Additional attributes like hostname, username, wifi password or ssh access key can be added
Installing Kubernetes pre-requisites
To get started and install K3s we need to install docker to the hosts. This can be done with the following commands
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the repository to Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
# Install latest docker version
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# Install specific version
apt-cache madison docker-ce | awk '{ print $3 }'
5:24.0.0-1~ubuntu.22.04~jammy
5:23.0.6-1~ubuntu.22.04~jammy
...
VERSION_STRING=5:24.0.0-1~ubuntu.22.04~jammy
sudo apt-get install docker-ce=$VERSION_STRING docker-ce-cli=$VERSION_STRING containerd.io docker-buildx-plugin docker-compose-plugin
Installing K3s with IPv6 support
K3s can be simply installed via a single line command. But in our case since we are behind a CGNAT (no native IPv4) we want to expose deployed services via IPv6
# Define custom K3S parameter to enable IPv6 support and priotize IPv6 traffic
INSTALL_K3S_EXEC=server --kubelet-arg=node-ip=:: --cluster-cidr=10.42.0.0/16,fd69::/48 --service-cidr=10.43.0.0/16,fd69::/112 --node-ip=<raspberry-primary-ipv4>,<raspberry-routable-ipv6>
# Invoke the K3S installation
curl -sfL https://get.k3s.io | sh -
# Get Agent installation token for the worker node
cat /var/lib/rancher/k3s/server/agent-token
K10877.....::server:....f12323
This step needs to be done only on the primary Kubernetes nodes which will be the only master node in the cluster. For all other nodes we need to change some values to register as a worker node.
# Define K3S installation parameter for the agent node
export INSTALL_K3S_EXEC="--kubelet-arg=node-ip=::"
# Invoke the K3S Agent installation
curl -sfL https://get.k3s.io | K3S_URL=https://[fd00::e65f:1ff:fe6c:a64b]:6443 K3S_TOKEN=K10877.....::server:....f12323 sh -
After this step a K3S Kubernetes cluster should be created and contain two nodes
# Execute on the master node
k3s kubectl get nodes
NAME STATUS ROLES AGE VERSION
rpi-k8s-node00 Ready control-plane,master 5h30m v1.28.4+k3s2
rpi-k8s-node01 Ready <none> 5h20m v1.28.4+k3s2
# Display running pods
k3s kubectl --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-6799fbcd5-qsxz4 1/1 Running 0 5h30m fd69:0:0:1::4 rpi-k8s-node00 <none> <none>
kube-system local-path-provisioner-84db5d44d9-mmt2p 1/1 Running 0 5h30m fd69:0:0:1::3 rpi-k8s-node00 <none> <none>
kube-system metrics-server-67c658944b-zzhm2 1/1 Running 0 5h30m fd69:0:0:1::5 rpi-k8s-node00 <none> <none>
kube-system helm-install-traefik-crd-v86xk 0/1 Completed 0 5h30m fd69:0:0:1::6 rpi-k8s-node00 <none> <none>
kube-system helm-install-traefik-l8jfj 0/1 Completed 2 5h30m fd69:0:0:1::2 rpi-k8s-node00 <none> <none>
kube-system svclb-traefik-f44b60c8-tjwdz 2/2 Running 0 5h29m fd69:0:0:1::7 rpi-k8s-node00 <none> <none>
kube-system traefik-f4564c4f4-r6hjz 1/1 Running 0 5h29m fd69:0:0:1::8 rpi-k8s-node00 <none> <none>
kube-system svclb-traefik-f44b60c8-5lgjm 2/2 Running 0 5h21m fd69:0:0:2::2 rpi-k8s-node01 <none> <none>
Installing Cert Manager with DNS challenge
To obtain valid Let's Encrypt certificates we need to add cert-manager. The installation will be done with helm and their predefined chart
# Add cert-manager repository
helm repo add jetstack https://charts.jetstack.io
# Update the repository
helm repo update
# Install cert-manager
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true --set 'extraArgs={--dns01-recursive-nameservers-only,--dns01-recursive-nameservers=8.8.8.8:53,1.1.1.1:53}'
Now we need to add all needed information to obtain certificates via DNS challenge from Cloudflare
apiVersion: v1
kind: Secret
metadata:
name: cloudflare-api-token-secret
namespace: cert-manager
type: Opaque
stringData:
api-token: BKW_XXXXXXXXXXXXXXXXXXXXX
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: le-global-issuer
spec:
acme:
email: me@example.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-key
solvers:
- dns01:
cloudflare:
email: me@example.com
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
---
Kubernetes persistent storage: NFS
Since we do not want to save all data onto the SDCards on the raspberry we add an additional storage class. This will be done with a external NFS server and a provisioner.
# Create a file called nfs and paste the following manifest
nano nfs
# Install NFS provisioner via manifest with all needed information
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: nfs
namespace: default
spec:
chart: nfs-subdir-external-provisioner
repo: https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
targetNamespace: default
set:
nfs.server: nfs-server-ip6
nfs.path: /volume2/swarm-nfs/rpik3s
storageClass.name: nfs
---
# Apply the helm chart
k3s kubectl apply nfs
Installing Ghost Blog
After all pre-requisites are done we can finally deploy our Ghost instance to the Raspberry Pi cluster. The software will be deployed via Bitnami Helm Chart. Some additional configurations needs to be done.
# create a new file: custom.yaml
global:
storageClass: "nfs"
image:
registry: docker.io
repository: bitnami/ghost
tag: 5.75.2
pullPolicy: IfNotPresent
ghostUsername: ghost-admin
ghostPassword: "beep_boop_top_secret"
ghostEmail: me@example.com
ghostBlogTitle: Cawfee & Tech
ghostEnableHttps: false
#smtpHost: ""
#smtpPort: ""
#smtpUser: ""
#smtpPassword: ""
#smtpService: ""
#smtpProtocol: ""
ghostSkipInstall: false
livenessProbe:
enabled: false
readinessProbe:
enabled: false
replicaCount: 1
updateStrategy:
type: Recreate
containerPorts:
http: 2368
https: 2368
service:
type: LoadBalancer
ports:
http: 80
https: 443
persistence:
enabled: true
storageClass: "nfs"
accessModes:
- ReadWriteOnce
size: 8Gi
volumePermissions:
enabled: true
ingress:
enabled: true
hostname: blog.cawfee.site
path: /
annotations:
service.beta.kubernetes.io/do-loadbalancer-hostname: "blog.cawfee.site"
cert-manager.io/cluster-issuer: "le-cs-issuer"
ingressClassName: "traefik"
extraTls:
- hosts:
- blog.cawfee.site
secretName: ghost.cawfee-site
mysql:
enabled: true
architecture: standalone
auth:
rootPassword: "some_secret_root_pw"
database: bitnami_ghost
username: bn_ghost
password: "beep_boop_mysql_user_password"
primary:
persistence:
enabled: true
storageClass: "nfs"
accessModes:
- ReadWriteOnce
size: 8Gi
---
# apply the helm chart with the custom values
helm upgrade --install --namespace blog --create-namespace blog -f custom.yaml oci://registry-1.docker.io/bitnamicharts/ghost
The end? nope!
Do not forget to open the ports to your Kubernetes nodes or the website will not be reachable from outside!