Building a Kubernetes (K3) Sandbox

I could be bounded in a nutshell and count myself master of infinite space –Macbeth.

Motivation

It all started with a desire, an itch to better understand the smallest hardware I could run an AI agent on. I started with a raspberry pi 4b and quickly learned that even an LLM the size of 394 MB ( qwen:0.5b) generates an out of memory exception when hosted locally on a computer the size of a credit card. Eventually, I decided I needed a larger shell for my kernel and built a kubernetes cluster which serves as a sandbox for explorations of this nature.

I chose a locally hosted kubernetes cluster because when I did the math:

• it is relativly low cost compared to commercially available enterprise solutions.
• it is an opportuny to learn about kubernetes and containerised applications
• running a cluster in my local network offers speed and isolation which is perfect for a sandbox
• it is an opportunity to explore the “code complete” capability of commercially available generative models such as Gemini and Claude.
• it is an opportunity to explore open source frameworks for Data Science and MLOps

Hardware / Setup

flowchart TB
    A("Control Plane (Mini PC)") --> B("Raspberry Pi 4B (Tainted Edge)")
    A --> C("Raspberry Pi 4B (Tainted Edge)")
    A --> E("Musashi (GPU node)")

• Control Plane: The control plane is a mini pc running linux. This routes traffic to all other nodes. [Done]
• Edge Nodes: The raspberry pi nodes are edge nodes running raspbian [Done]
• Msuashi: A GPU node for running LLMs and training models 95% Complete

Core platform services

• Container Registry: A MinIO deployment acting both as storage and a container registry
• Monitoring: A Grafana deployment for monitoring nodes
• GitOps: ArgoCD for streamlining deployment
• Git Repository: Forejo for versioning code and other artefact

Data Science services

• Ollama: A deployment that exposes LLMs downloadable from ollama via a web service
• Qdrant: A vector database for text analytics. This offers both storage and a UI for exploring the stored data.
• Langflow: A service for creating AI workflows via visual interface
• Prefect: Enables the building of data pipelines
• MQTT: For streaming live data from edge nodes (raspberry pi 4B)

Incubation workflow

flowchart LR
    A("An Itch/A need") --> B[["Experimentation/Ideation"]]
    B --> C("Insight / Outcome")
    C --> D{"Repetitive & Generic"}
    D--> E("Yes")
    D--> F("No")
    E--> G("Deploy")

It starts with an itch, a desire to understand which then leads to playful experimentation that generates anecdotal isights that enable me to decide whether or not I should deploy the process as an app.

Lessons learned

• Commerical Generative models like Opus (Claude) and Gemini (3.1) a force multiplier for MLOps productivity
• A homelab cluster can be a cost effective sandox for ideation
• K3, a lightweight kubernetes is quite effective for deploying services - on average less than an hour to deploy a service.

Next steps

• Setup the GPUs on Musashi as a resource in the K3 cluster