Sharing my Development K8s Setup

,

Hello, this is my blog first entry! And as a kubernetes enthusiast, my first entry, of course, is about how I spin up my sandbox everyday. I do a lot of experiments with k8s and I often spin up fresh cluster to work on after each experiment.

In this entry, I will discuss what platform I’m using in spinning up a cluster for different kinds of scenario, and how I make these reprovisioning a cluster a breeze.

So what’s my daily workhorse?

If what I’ll work on doesn’t require a lot of resource, like CPU or RAM, I usually just work on my laptop. And the tool I use is KinD (Kubernetes in Docker).

Why use KinD? And not minikube or docker desktop? There are 2 basic reasons why I use KinD -

Pre-requisites

  • Container runtime, I’m using Docker.
  • KinD installed - read here to learn more on how to install KinD.

Fast setup and cleanup

First, spinning up cluster on KinD is very fast.

It’s as easy as:

1
kind create cluster --name cluster1

By default, it spins up a 1 node cluster. It also automatically loads the kubeconfig for you so you can access your cluster rightaway. You can spin up mutiple cluster with KinD, if you have enough resources for it.
This is not possible with minikube or docker desktop. Yes, most of the time, 1 cluster is enough to do basic development work. But if you need to spin up multiple clusters, at least you have this option.

If you think that a 1 node cluster is what you’ll need forever, then I suggest to check Rancher Desktop, the built-in kubernetes here is provided by k3s which means its light weight. This is a good replacement for docker desktop now that it’s slowly becoming a proprietary software.

When you’re done with what you’re doing and it’s time to clean up, just:

1
kind delete cluster --name cluster1

And your cluster will be deleted in a minute. Very fast, right?

By the way, if you want to learn more on how to expose your deployed apps on KinD using ingress read more here.

Multi node in KinD?

The second reason I use KinD is it lets me spin up a multinode cluster. This is not possible with minikube or docker desktop as well. Again, a 1-node cluster can get the job done when doing some dev work, but there are times when I need to simulate scenarios where multiple nodes are involved - like when testing fault tolerance of some tools.

To provision a multi-node, you have to create a KinD config file. It looks something like this:

1
2
3
4
5
6
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
- role: control-plane
- role: worker
- role: worker

In this KinD manifest, we configured our cluster to have 1 master and 2 worker nodes.

To create the cluster, just:

1
kind create cluster --name cluster1 --config kind-config.yaml

And this will spin up the multi-node cluster that you need.

Using KinD manifest is not limited to specifying the number of nodes in your cluster. You can also configure cluster-wide options (like runtime configs, networking and feature gates) and Node specific configs.

1
2
3
4
5
6
7
8
9
10
11
12
13
kind: Cluster
networking:
  podSubnet: "10.10.0.0/16"
  serviceSubnet: "10.20.0.0/12"
featureGates:
  "GracefulNodeShutdown": true
nodes:
- role: control-plane
  image: kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
  extraMounts:
  - hostPath: /path/to/my/files/
    containerPath: /files
    readOnly: true

And thats it for KinD!

What if I need more muscle?

If I know that I’ll need bigger muscles for my project, of course, cloud is your friend!
My provider of choice is GCP, as provisioning GKE is the simplest among the bunch. The tool I’m using to automate this process is terraform. You can visit my repo here to learn more about the IaaC that I’m using.

And that’s all I have, you guys (and gals)! Thank you for reading my entry until the end. Until next time!