Intro

K3s is a lightweight and top-tier choice for running Kubernetes in on-premise, edge computing, and IoT environments. It's a CNCF-certified distribution that is also easy to set up. Although it's lightweight, K3s is fully production-ready.

By default, it utilizes SQLite for its datastore, which is centralized on a single master node with a single control plane. This architecture is not designed for the high-concurrency and distributed nature of large-scale Kubernetes clusters. Consequently, if the master node goes down, the cluster becomes unmanageable and cannot be updated or self-heal.

Solution

To achieve High Availability (HA) in K3s, we must replace the default architecture. This involves using either an embedded etcd datastore or an external datastore, distributing the control plane across multiple nodes, and making the entire setup fault-tolerant.

External datastore

An external datastore in K3s is a database that stores all cluster's state and configuration, but it runs completely separate from the K3s server (master) nodes.

This decouples the cluster's "brain" (the control plane) from its "memory" (the datastore). Instead of K3s managing its own database internally, it's configured to connect to your pre-existing, dedicated database system.

External datastore options are etcd, postgres, mysql, and so on.

Internal datastore

An internal datastore in K3s means the database that stores all the cluster's configuration and state runs directly on the server (master) nodes themselves.

Internal datastore options are SQLserver (default, single control plane) and embedded etcd.

Which should we use?

If we want to self-contained, production-ready clusters where simplicity is a priority. go with embedded etcd but if we want to large-scale, critical production clusters needing independent scaling and reliability then go with external datastore.