(Part 1) A Kubernetes of One's Own: I Can Haz Cluster?
In my last blog post, I hope I convinced you why you should be creating your own Kubernetes cluster for personal usage. Now, we can tackle the fun part of creating the cluster.
What is a Kubernetes cluster?
We can start by answering the most important question: what is a Kubernetes cluster? A Kubernetes cluster is a collection of physical resources on which we run the Kubernetes container management software. Once we have a Kubernetes cluster, we can use it to host applications, run batch workloads, etc.
The methods for creating and managing a Kubernetes cluster fall into a couple of distinct categories.
First, we have local-machine solutions. In this model, we use our local workstations resources to run Kubernetes. Minikube is the most popular example of this pattern.
Second, we have hosted solutions. In this model, a cloud provider Google both provides the physical resources and manages the Kubernetes software for us. Our job is purely application specific configuration and code.
Third, we have turnkey cloud solutions, which allow us to easily create a Kubernetes cluster on common cloud platforms with only a couple commands. These turnkey solutions also often provide tooling for managing the Kubernetes cluster once its running. kops is popular tooling supporting a turnkey solution.
Fourth, we have on-premises turnkey solutions. These solutions are similar to the turnkey cloud solutions, but instead of creating a Kubernetes cluster in the public cloud (i.e. AWS, GCE), they create it on your own internal cloud network.
Finally, for the truly bold, there are custom solutions, which essentially leave all of the installation, configuration, and management to the user.
Regardless of how we create the cluster, all Kubernetes clusters share foundational elements. With respect to hardware, a cluster contains both a master node and a set of worker nodes. With respect to software, it has six main components: API server, scheduler, controller manager, kubelet, kube-proxy, and etcd. Julia Evans has a fantastic blog post which does a much better job than I could explaining what these components do.
What are our goals for our Kubernetes cluster?
Because each method for creating a Kubernetes cluster has different strengths and weaknesses, we need to think about what we value in the cluster creation and maintenance process. At a high level, I have the following priorities in order of importance:
- Initial creation of the cluster should be painless. We want to start using Kubernetes now. We don't want to spend weeks and weeks wrestling with the cluster creation process.
- The cluster creation and management process provides the right level of abstraction. Kubernetes clusters have many different knobs you can turn. I'd like access to these knobs, but I would also like them to start at sensible values.
- I want to pay as little as possible.
- Because I'm most familiar with AWS and use it professionally, I'd like to use a cluster creation process which supports AWS, but isn't necessarily restricted to AWS.
Evaluating the options
With our valuation framework in place, we can start honing in on the perfect option for us for creating a Kubernetes cluster.
Remember, Kubernetes offers local-machine, hosted, turnkey cloud, on-premises turnkey, and custom solutions for creating a Kubernetes cluster. Since we want our cluster to fulfill production use cases, local-machine solutions are not feasible. Since we want to pay as little as possible and I don't have multiple spare servers just lying around, on-premises turnkey solutions are out. And since I want setting up the cluster to not be an absolute nightmare, custom solutions are out.
After eliminating what won't work, we are left to decide between hosted and turnkey cloud solutions. Considering our goal of using AWS as our cloud provider, we are deciding between Amazon EKS and using kops to create and manage our Kubernetes cluster on AWS public cloud.
After some experimentation, I decided on Kops. Kops offers sensible defaults, yet also fine grained control. Additionally, it imposes no costs beyond the AWS resources it uses. And most importantly, from reading quick start documentation for EKS and Kops, I felt more confident I could successfully create a cluster and manage a cluster using Kops than EKS.
So that's it! I choose Kops for creating and managing our cluster! In the next blog post, I'll walk you through all the steps you need to create your cluster. Looking forward to it :)