Building a Custom Kubernetes Console

After abandoning KubeSphere due to installation issues, I built a custom web console to manage my homelab. This article covers the architecture, implementation, and lessons learned.

Why Build a Custom Console?

KubeSphere Issues:

• Installation failures with outdated Helm charts
• Image pull errors
• Complex dependencies
• Overkill for a single-node lab

Custom Solution Benefits:

• Lightweight and fast
• Exactly what we need, nothing more
• Full control over features
• Learning opportunity

Architecture

┌─────────────────────────────────────┐
│   React Frontend (TypeScript)      │
│   - Dashboard                       │
│   - Nodes View                      │
│   - Pods View                       │
│   - VMs View                        │
└──────────────┬──────────────────────┘
               │ HTTP/REST
┌──────────────▼──────────────────────┐
│   Go Backend (REST API)              │
│   - Kubernetes client-go              │
│   - Dynamic client for KubeVirt        │
│   - CORS middleware                   │
└──────────────┬──────────────────────┘
               │
┌──────────────▼──────────────────────┐
│   Kubernetes API Server              │
│   - Nodes, Pods, Services            │
│   - KubeVirt VirtualMachines         │
└──────────────────────────────────────┘

Backend: Go + client-go

Architecture Decisions

Why Go?

• Excellent Kubernetes client libraries
• Fast compilation and execution
• Simple deployment (single binary)
• Good concurrency support

Why Dynamic Client for KubeVirt?

• Avoids version conflicts between k8s.io/client-go and kubevirt.io/client-go
• More flexible for CRDs
• Easier to maintain

Implementation

Main Server (main.go):

type Server struct {
    k8sClient     *kubernetes.Clientset
    dynamicClient dynamic.Interface
}

func main() {
    // Initialize Kubernetes clients with retry logic
    config, err := getKubeConfig()
    k8sClient, _ := kubernetes.NewForConfig(config)
    dynamicClient, _ := dynamic.NewForConfig(config)
    
    server := &Server{
        k8sClient:     k8sClient,
        dynamicClient: dynamicClient,
    }
    
    // Setup routes
    r := mux.NewRouter()
    api := r.PathPrefix("/api/v1").Subrouter()
    api.HandleFunc("/cluster/nodes", server.getNodes)
    api.HandleFunc("/cluster/pods", server.getPods)
    api.HandleFunc("/vms", server.getVMs)
    // ... more routes
}

Key Features:

1. Retry Logic for Kubeconfig

   • Handles cases where the app starts before Kubernetes is ready
   • Tries in-cluster config first, falls back to kubeconfig file
   • Exponential backoff for resilience

2. Dynamic Client for KubeVirt

   var vmGVR = schema.GroupVersionResource{
       Group:    "kubevirt.io",
       Version:  "v1",
       Resource: "virtualmachines",
   }
   
   vms, err := s.dynamicClient.Resource(vmGVR).List(context.TODO(), metav1.ListOptions{})
   

3. VM Control Operations

   • Start/Stop/Restart VMs by updating spec.running
   • Create/Delete VMs via dynamic client
   • Full CRUD operations

API Endpoints:

GET  /api/v1/cluster/nodes
GET  /api/v1/cluster/pods?namespace=default
GET  /api/v1/cluster/services
GET  /api/v1/vms?namespace=default
GET  /api/v1/vms/{namespace}/{name}
POST /api/v1/vms
POST /api/v1/vms/{namespace}/{name}/start
POST /api/v1/vms/{namespace}/{name}/stop
POST /api/v1/vms/{namespace}/{name}/restart
DELETE /api/v1/vms/{namespace}/{name}

Frontend: React + TypeScript

Tech Stack

• React 18 - UI framework
• TypeScript - Type safety
• Vite - Fast build tool
• TailwindCSS - Utility-first styling
• React Query - Data fetching and caching
• React Router - Client-side routing
• Axios - HTTP client

Component Structure

src/
├── App.tsx              # Main router
├── pages/
│   ├── Dashboard.tsx    # Overview with stats
│   ├── Nodes.tsx        # Node details
│   ├── Pods.tsx         # Pod listing and filtering
│   └── VirtualMachines.tsx  # VM management
└── index.css            # Tailwind imports

Key Features

1. Real-time Data Fetching

const { data: pods } = useQuery({
  queryKey: ['pods', namespace],
  queryFn: async () => {
    const { data } = await axios.get(`/api/v1/cluster/pods?namespace=${namespace}`)
    return data
  },
  refetchInterval: 5000, // Auto-refresh every 5s
})

2. VM Control Actions

const startVM = async (namespace: string, name: string) => {
  await axios.post(`/api/v1/vms/${namespace}/${name}/start`)
  queryClient.invalidateQueries(['vms'])
}

3. Namespace Filtering

const namespaces = ['default', 'blog', 'npm', 'kube-system']
const [selectedNamespace, setSelectedNamespace] = useState('all')

Deployment

Docker Images

Backend Dockerfile:

FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /tom-lab-console

FROM alpine:latest
COPY --from=builder /tom-lab-console /tom-lab-console
CMD ["/tom-lab-console"]

Frontend Dockerfile:

FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: console-api
  namespace: console
spec:
  replicas: 1
  template:
    spec:
      serviceAccountName: console-api
      containers:
      - name: api
        image: 192.168.1.100:30500/tom-lab-console-api:latest
        ports:
        - containerPort: 8080
        env:
        - name: PORT
          value: "8080"
---
apiVersion: v1
kind: Service
metadata:
  name: console-api
spec:
  type: ClusterIP
  ports:
  - port: 8080
    targetPort: 8080

Challenges & Solutions

1. Architecture Mismatch

Problem: Built on Apple Silicon (ARM64), deployed on AMD64 server.

Solution:

RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build

And in build script:

docker build --platform linux/amd64 -t $REGISTRY/tom-lab-console-api:latest .

2. KubeVirt Client Version Conflicts

Problem: kubevirt.io/client-go had incompatible versions with k8s.io/client-go.

Solution: Use dynamic client instead:

import "k8s.io/client-go/dynamic"

3. In-Cluster Config Timing

Problem: App starts before service account token is available.

Solution: Retry logic with exponential backoff:

for i := 0; i < 10; i++ {
    config, err = rest.InClusterConfig()
    if err == nil {
        break
    }
    time.Sleep(time.Duration(i+1) * time.Second)
}

4. Local Registry Configuration

Problem: Talos trying to pull from local registry via HTTPS.

Solution: Patch Talos config:

machine:
  registries:
    config:
      192.168.1.100:30500:
        protocol: http
        tls:
          insecureSkipVerify: true

Features

Dashboard

• Cluster overview with key metrics
• Node status and resources
• Running pods count
• Active VMs count
• Recent activity

Nodes View

• Node details (CPU, memory, disk)
• Conditions and status
• Labels and annotations
• Resource usage

Pods View

• Filter by namespace
• Search by name
• Status indicators
• Quick actions (logs, describe)

VMs View

• List all VirtualMachines
• Start/Stop/Restart actions
• Status monitoring
• Create new VMs (future)

Future Enhancements

• VM creation wizard
• Pod logs viewer
• Resource usage graphs
• Event timeline
• Multi-cluster support
• Authentication/Authorization
• Dark/light theme toggle

Lessons Learned

1. Start Simple - Custom console is simpler than KubeSphere for our needs
2. Dynamic Client is Powerful - Avoids version conflicts, more flexible
3. Architecture Matters - Always build for target platform
4. Retry Logic is Essential - Kubernetes startup timing is unpredictable
5. React Query is Great - Simplifies data fetching and caching

Code

Full source code available in web-console/ directory:

• Backend: web-console/backend/
• Frontend: web-console/frontend/
• Deployment: web-console/deploy/

Next: Nginx Proxy Manager Setup