Kubernetes Setup¶

This guide installs a minimal k0s cluster (lab-cluster) on master01 and worker01 with k0sctl, then hands the cluster over to Flux, which installs every add-on from riupie/gitops-fluxcd.

Prerequisites¶

• Prerequisites and Infrastructure Deployment completed

• VMs running and reachable over SSH as cloud (the ssh_admin user created by the VM module):

  virsh list
  ssh cloud@192.168.10.10 "hostname && uptime"
  ssh cloud@192.168.10.11 "hostname && uptime"
  

• kubectl installed on the host: kubectl version --client

Steps¶

1. Install k0sctl¶

On the host. Download the release binary and verify its checksum:

K0SCTL_VERSION=v0.33.1
dl="$(mktemp -d)" && cd "$dl"
curl -fsSLO https://github.com/k0sproject/k0sctl/releases/download/${K0SCTL_VERSION}/k0sctl-linux-amd64
curl -fsSLO https://github.com/k0sproject/k0sctl/releases/download/${K0SCTL_VERSION}/checksums.txt
sha256sum --check --ignore-missing checksums.txt
sudo install -m 0755 k0sctl-linux-amd64 /usr/local/bin/k0sctl
k0sctl version
cd - >/dev/null && rm -rf "$dl"

2. Review the cluster configuration¶

The configuration is k0s/k0sctl.yaml in this repo:

3. Deploy the cluster¶

On the host, from the repo root:

k0sctl apply --config k0s/k0sctl.yaml

Add --debug if a step fails. k0sctl checks SSH access, installs k0s on both nodes, initializes the controller, joins the worker and installs the flux-operator chart.

4. Get cluster access¶

mkdir -p ~/.kube
k0sctl kubeconfig --config k0s/k0sctl.yaml > ~/.kube/config
kubectl config current-context    # lab-cluster
kubectl --context lab-cluster cluster-info

5. Connect Flux to the GitOps repository¶

flux-operator is running but has nothing to sync yet. Create the FluxInstance that points it at gitops-fluxcd:

kubectl --context lab-cluster apply -f - <<'EOF'
apiVersion: fluxcd.controlplane.io/v1
kind: FluxInstance
metadata:
  name: flux
  namespace: flux-system
spec:
  distribution:
    version: "2.x"
    registry: ghcr.io/fluxcd
    artifact: oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests
  components:
    - source-controller
    - kustomize-controller
    - helm-controller
    - notification-controller
  cluster:
    type: kubernetes
    size: small
    networkPolicy: true
  sync:
    kind: GitRepository
    url: https://github.com/riupie/gitops-fluxcd.git
    ref: refs/heads/main
    path: clusters/development
    interval: 1m
EOF

From here on, Flux reconciles every add-on from clusters/development in the repo: change the add-ons by committing to gitops-fluxcd, not with kubectl apply. The kubectl apply above only bootstraps the FluxInstance. Once it syncs, Flux also reconciles the copy in clusters/development/flux-system/flux-instance.yaml, so change it there rather than re-applying the manifest above.

Verify¶

1. Nodes and system pods¶

kubectl --context lab-cluster get nodes -o wide
kubectl --context lab-cluster -n kube-system get pods

Expected output:

NAME       STATUS   ROLES    AGE   VERSION       INTERNAL-IP     ...
worker01   Ready    <none>   4m    v1.36.4+k0s   192.168.10.11   ...

kube-system contains calico-*, coredns-*, konnectivity-agent-*, kube-proxy-* and metrics-server-* pods, all Running.

2. Flux¶

kubectl --context lab-cluster -n flux-system get fluxinstance,gitrepository,kustomization
kubectl --context lab-cluster get helmrelease -A

Expected: the FluxInstance flux is READY True, the flux-system GitRepository points at https://github.com/riupie/gitops-fluxcd.git, and the Kustomizations (flux-system, infrastructure, infrastructure-configs, apps) are READY True. HelmReleases for MetalLB, cert-manager, External-DNS, External Secrets, Sealed Secrets and agentgateway appear as they reconcile.

3. MetalLB¶

kubectl --context lab-cluster -n metallb get ipaddresspool,l2advertisement

Expected: IPAddressPool lb-pool with 192.168.10.100-192.168.10.110 and a matching L2Advertisement.

Troubleshooting¶

Cleanup¶

⚠ WARNING: Destructive operation. This removes k0s and all cluster data (etcd, workloads, PVC contents on worker01) from both nodes. The VMs are left in place.

k0sctl reset --config k0s/k0sctl.yaml    # asks for confirmation; add --force to skip it

Then remove the local access you created in step 4 (this deletes ~/.kube/config entirely; skip it if the file holds other clusters):

rm ~/.kube/config

To remove the VMs as well, follow Infrastructure Deployment, Cleanup. Removing the VMs also removes the cluster, so k0sctl reset is only needed to rebuild the cluster on the same VMs.

Next steps¶

1. Set Up Dynamic DNS: BIND9 on general01, which External-DNS writes to
2. Secure MCP Servers with OAuth: Keycloak in front of MCP servers on agentgateway