Skip to main content
Catskills Digital

Feature

A First On-Prem Kubernetes Cluster Installation on Ubuntu

The easiest way to make Kubernetes difficult is to begin with the architecture you eventually want. Multiple nodes, redundant storage, automated failover, GitOps, remote disaster recovery and hybrid infrastructure are all reasonable things to think about eventually. They also introduce enough interacting parts that a first installation can fail in several p

The Mountain Eagle

The easiest way to make Kubernetes difficult is to begin with the architecture you eventually want.

Multiple nodes, redundant storage, automated failover, GitOps, remote disaster recovery and hybrid infrastructure are all reasonable things to think about eventually. They also introduce enough interacting parts that a first installation can fail in several places at once.

I'd rather start with one machine.

Not because one machine is a good production architecture for everything, but because it gives us a small enough environment to understand what Kubernetes is actually doing before we ask it to coordinate anything else.

Assume a fresh Ubuntu 24.04 LTS installation on a physical server or virtual machine we control. It has a predictable IP address, working DNS, SSH access and Internet connectivity for retrieving packages and container images.

Before installing Kubernetes, I want the operating system itself uneventful.

Update it. Make sure DNS resolution works. Confirm that the clock is synchronized. Decide how security updates will be handled. Verify that the machine returns cleanly after a reboot.

If the Kubernetes distribution or configuration we're using has particular requirements around swap, networking, firewall rules or kernel settings, deal with those explicitly rather than applying a collection of commands copied from an unrelated installation guide.

The objective isn't to make Ubuntu special.

It's to establish a known starting point.

For this build, I'm using k3s.

There are many legitimate ways to run Kubernetes, and k3s makes a set of decisions for us that would otherwise have to be made individually. It packages the Kubernetes components into a comparatively compact distribution, provides an installation and service-management path, and includes defaults intended to make a small cluster usable without assembling every component ourselves.

Those defaults matter because installing k3s isn't the same thing as installing a completely empty upstream Kubernetes environment.

That's worth understanding before changing them.

A normal k3s installation gives us a server running the Kubernetes control plane and datastore along with the components necessary for the node to run workloads. Depending on the version and installation options, k3s also includes several packaged components and defaults, including networking and an ingress controller.

So after installation, my first question isn't what else I can install.

It's what just appeared.

Check the service.

Run kubectl get nodes.

Look at the pods running across the namespaces.

Find the kubeconfig.

See which components k3s installed and which ones are merely Kubernetes concepts we've read about but aren't actually present in this cluster.

Then reboot the machine.

This is probably the least impressive Kubernetes test we'll ever perform, which is one reason I like doing it immediately.

When the server returns, does k3s start?

Does the node become Ready again?

Do the system workloads recover?

Can kubectl talk to the cluster?

If something doesn't return, now is a very good time to understand why. There is one machine, almost no application state and relatively little configuration to investigate.

We're buying cheap failures while they're still available.

Once the basic cluster survives a reboot, I want to put one disposable application on it.

Nothing important.

Run a tiny web service whose only job is to return a recognizable response. At first we can reach it from inside the cluster and verify that the service abstraction points toward the expected pod.

Then we can work outward.

This is where ingress enters the picture, although with k3s we first need to inspect the ingress capability already installed.

A default k3s installation commonly includes Traefik. If that meets the requirements, there's no particular reason to replace it merely because another ingress controller is more familiar.

If we've deliberately decided to use something else, such a