Skip to content

Commit

Permalink
Improve project description and documentation (#365)
Browse files Browse the repository at this point in the history
* docs: improve documentation

* docs: improve documentation

* docs: improve documentation
  • Loading branch information
bsctl authored Aug 29, 2023
1 parent 973392b commit bac5d56
Show file tree
Hide file tree
Showing 17 changed files with 92 additions and 85 deletions.
39 changes: 9 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@
![Logo](assets/logo-black.png#gh-light-mode-only)
![Logo](assets/logo-white.png#gh-dark-mode-only)

**Kamaji** deploys and operates **Kubernetes Control Plane** at scale with a fraction of the operational burden. Kamaji is special because the Control Plane components are running in a single pod instead of dedicated machines. This solution makes running multiple Control Planes cheaper and easier to deploy and operate.
**Kamaji** is a **Kubernetes Control Plane Manager**. It operates Kubernetes at scale with a fraction of the operational burden. Kamaji is special because the Control Plane components are running inside pods instead of dedicated machines. This solution makes running multiple Control Planes cheaper and easier to deploy and operate.

<img src="docs/content/images/architecture.png" width="600">

## Features
## Main Features

- **Self Service Kubernetes:** leave users the freedom to self-provision their Kubernetes clusters according to the assigned boundaries.
- **Multi-cluster Management:** centrally manage multiple clusters from a single admin cluster. Happy SREs.
- **Cheaper Control Planes:** place multiple control planes on a single node, instead of having three nodes for a single control plane.
- **Stronger Multi-Tenancy:** leave users to access the control plane with admin permissions while keeping them isolated at the infrastructure level.
- **Kubernetes Inception:** use Kubernetes to manage Kubernetes by re-using all the Kubernetes goodies you already know and love.
- **Full APIs compliant:** all clusters are CNCF compliant built with upstream Kubernetes binaries
- **Multi-cluster Management:** centrally manage multiple Kubernetes clusters from a single Management Cluster.
- **High-density Control Plane:** place multiple control planes on the same infrastructure, instead of having dedicated machines for each control plane.
- **Strong Multi-tenancy:** leave users to access the control plane with admin permissions while keeping them isolated at the infrastructure level.
- **Kubernetes Inception:** use Kubernetes to manage Kubernetes with automation, high-availability, fault tolerance, and autoscaling out of the box.
- **Bring Your Own Device:** keep the control plane isolated from data plane. Worke nodes can join and run consistently everywhere: cloud, edge, and data-center.
- **Full CNCF compliant:** all clusters are built with upstream Kubernetes binaries, resulting in full CNCF compliant Kubernetes clusters.

## Roadmap

Expand All @@ -37,32 +37,11 @@
- [ ] Autoscaling of Tenant Control Plane
- [x] Provisioning through Cluster APIs
- [ ] Terraform provider
- [ ] Custom Prometheus metrics for monitoring and alerting
- [ ] Custom Prometheus metrics


## Documentation
Please, check the project's [documentation](https://kamaji.clastix.io/) for getting started with Kamaji.

## Contributions
Kamaji is Open Source with Apache 2 license and any contribution is welcome. Open an issue or suggest an enhancement on the GitHub [project's page](https://github.com/clastix/kamaji). Join the [Kubernetes Slack Workspace](https://slack.k8s.io/) and the [`#kamaji`](https://kubernetes.slack.com/archives/C03GLTTMWNN) channel to meet end-users and contributors.

## FAQs
Q. What does Kamaji mean?

A. Kamaji is named as the character _Kamaji_ from the Japanese movie [_Spirited Away_](https://en.wikipedia.org/wiki/Spirited_Away).

Q. Is Kamaji another Kubernetes distribution?

A. No, Kamaji is a Kubernetes Operator you can install on top of any Kubernetes cluster to provide hundreds or thousands of managed Kubernetes clusters as a service. We tested Kamaji on vanilla Kubernetes 1.22+, KinD, and Azure AKS. We expect it to work smoothly on other Kubernetes distributions. The tenant clusters made with Kamaji are conformant CNCF Kubernetes clusters as we leverage [`kubeadm`](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/).

Q. Is it safe to run Kubernetes control plane components in a pod instead of dedicated virtual machines?

A. Yes, the tenant control plane components are packaged in the same way they are running in bare metal or virtual nodes. We leverage the `kubeadm` code to set up the control plane components as they were running on their own server. The unchanged images of upstream `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager` are used.

Q. You already provide a Kubernetes multi-tenancy solution with [Capsule](https://capsule.clastix.io). Why does Kamaji matter?

A. A multi-tenancy solution, like Capsule shares the Kubernetes control plane among all tenants keeping tenant namespaces isolated by policies. While the solution is the right choice by balancing between features and ease of usage, there are cases where a tenant user requires access to the control plane, for example, when a tenant requires to manage CRDs on his own. With Kamaji, you can provide cluster admin permissions to the tenant.

Q. Well you convinced me, how to get a try?

A. It is possible to get started with Kamaji on a laptop with [KinD](getting-started.md) installed.
2 changes: 1 addition & 1 deletion charts/kamaji/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
appVersion: v0.3.3
description: Kamaji deploys and operates Kubernetes at scale with a fraction of the operational burden. Kamaji turns any Kubernetes cluster into an “admin cluster” to orchestrate other Kubernetes clusters called “tenant clusters”. Kamaji is special because the Control Plane components are running in a single pod instead of dedicated machines. This solution makes running multiple Control Planes cheaper and easier to deploy and operate.
description: Kamaji is a Kubernetes Control Plane Manager.
home: https://github.com/clastix/kamaji
icon: https://github.com/clastix/kamaji/raw/master/assets/logo-colored.png
kubeVersion: ">=1.21.0-0"
Expand Down
2 changes: 1 addition & 1 deletion charts/kamaji/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

![Version: 0.12.4](https://img.shields.io/badge/Version-0.12.4-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.3.3](https://img.shields.io/badge/AppVersion-v0.3.3-informational?style=flat-square)

Kamaji deploys and operates Kubernetes at scale with a fraction of the operational burden. Kamaji turns any Kubernetes cluster into an “admin cluster” to orchestrate other Kubernetes clusters called “tenant clusters”. Kamaji is special because the Control Plane components are running in a single pod instead of dedicated machines. This solution makes running multiple Control Planes cheaper and easier to deploy and operate.
Kamaji is a Kubernetes Control Plane Manager.

## Maintainers

Expand Down
22 changes: 11 additions & 11 deletions docs/content/concepts.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
# Concepts

Kamaji is a Kubernetes Operator. It turns any Kubernetes cluster into an _admin cluster_ to orchestrate other Kubernetes clusters called _tenant clusters_.
**Kamaji** is a **Kubernetes Control Plane Manager**. It operates Kubernetes at scale with a fraction of the operational burden. Kamaji turns any Kubernetes cluster into a _Management Cluster_ to orchestrate other Kubernetes clusters called _Tenant Clusters_.

These are requirements of the design behind Kamaji:

- Communication between the _admin cluster_ and a _tenant cluster_ is unidirectional. The _admin cluster_ manages a _tenant cluster_, but a _tenant cluster_ has no awareness of the _admin cluster_.
- Communication between different _tenant clusters_ is not allowed.
- Communication between the _Management Cluster_ and a _Tenant Cluster_ is unidirectional. The _Management Cluster_ manages a _Tenant Cluster_, but a _Tenant Cluster_ has no awareness of the _Management Cluster_.
- Communication between different _Tenant Clusters_ is not allowed.
- The worker nodes of tenant should not run anything beyond tenant's workloads.

Goals and scope may vary as the project evolves.

## Tenant Control Plane
Kamaji is special because the Control Planes of the _tenant cluster_ are regular pods running in a namespace of the _admin cluster_ instead of a dedicated set of Virtual Machines. This solution makes running Control Planes at scale cheaper and easier to deploy and operate. The Tenant Control Plane components are packaged in the same way they are running in bare metal or virtual nodes. We leverage the `kubeadm` code to set up the control plane components as they were running on their own server. The unchanged images of upstream `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager` are used.
Kamaji is special because the Control Planes of the _Tenant Clusters_ are regular pods running in a namespace of the _Management Cluster_ instead of a dedicated machines. This solution makes running Control Planes at scale cheaper and easier to deploy and operate. The Tenant Control Plane components are packaged in the same way they are running in bare metal or virtual nodes. We leverage the `kubeadm` code to set up the control plane components as they were running on their own server. The unchanged images of upstream `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager` are used.

High Availability and rolling updates of the Tenant Control Plane pods are provided by a regular Deployment. Autoscaling based on the metrics is available. A Service is used to espose the Tenant Control Plane outside of the _admin cluster_. The `LoadBalancer` service type is used, `NodePort` and `ClusterIP` are other viable options, depending on the case.
High Availability and rolling updates of the Tenant Control Plane pods are provided by a regular Deployment. Autoscaling based on the metrics is available. A Service is used to espose the Tenant Control Plane outside of the _Management Cluster_. The `LoadBalancer` service type is used, `NodePort` and `ClusterIP` are other viable options, depending on the case.

Kamaji offers a [Custom Resource Definition](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/) to provide a declarative approach of managing a Tenant Control Plane. This *CRD* is called `TenantControlPlane`, or `tcp` in short.

All the _tenant clusters_ built with Kamaji are fully compliant CNCF Kubernetes clusters and are compatible with the standard Kubernetes toolchains everybody knows and loves. See [CNCF compliance](reference/conformance.md).
All the _Tenant Clusters_ built with Kamaji are fully compliant CNCF Kubernetes clusters and are compatible with the standard Kubernetes toolchains everybody knows and loves. See [CNCF compliance](reference/conformance.md).

## Tenant worker nodes

And what about the tenant worker nodes?
They are just _"worker nodes"_, i.e. regular virtual or bare metal machines, connecting to the APIs server of the Tenant Control Plane.
Kamaji's goal is to manage the lifecycle of hundreds of these _tenant clusters_, not only one, so how to add another tenant cluster to Kamaji?
As you could expect, you have just deploys a new Tenant Control Plane in one of the _admin cluster_ namespace, and then joins the tenant worker nodes to it.
Kamaji's goal is to manage the lifecycle of hundreds of these _Tenant Clusters_, not only one, so how to add another Tenant Cluster to Kamaji?
As you could expect, you have just deploys a new Tenant Control Plane in one of the _Management Cluster_ namespace, and then joins the tenant worker nodes to it.

A [Cluster API ControlPlane provider](https://github.com/clastix/cluster-api-control-plane-provider-kamaji) has been released, allowing to offer a Cluster API-native declarative lifecycle, by automating the worker nodes join.

## Datastores
Putting the Tenant Control Plane in a pod is the easiest part. Also, we have to make sure each tenant cluster saves the state to be able to store and retrieve data. As we can deploy a Kubernetes cluster with an external `etcd` cluster, we explored this option for the Tenant Control Planes. On the admin cluster, you can deploy one or multi-tenant `etcd` to save the state of multiple tenant clusters. Kamaji offers a Custom Resource Definition called `DataStore` to provide a declarative approach of managing multiple datastores. By sharing the datastore between multiple tenants, the resiliency is still guaranteed and the pods' count remains under control, so it solves the main goal of resiliency and costs optimization. The trade-off here is that you have to operate external datastores, in addition to `etcd` of the _admin cluster_ and manage the access to be sure that each _tenant cluster_ uses only its data.
Putting the Tenant Control Plane in a pod is the easiest part. Also, we have to make sure each Tenant Cluster saves the state to be able to store and retrieve data. As we can deploy a Kubernetes cluster with an external `etcd` cluster, we explored this option for the Tenant Control Planes. On the Management Cluster, you can deploy one or multi-tenant `etcd` to save the state of multiple Tenant Clusters. Kamaji offers a Custom Resource Definition called `DataStore` to provide a declarative approach of managing multiple datastores. By sharing the datastore between multiple tenants, the resiliency is still guaranteed and the pods' count remains under control, so it solves the main goal of resiliency and costs optimization. The trade-off here is that you have to operate external datastores, in addition to `etcd` of the _Management Cluster_ and manage the access to be sure that each _Tenant Cluster_ uses only its data.

### Other storage drivers
Kamaji offers the option of using a more capable datastore than `etcd` to save the state of multiple tenants' clusters. Thanks to the native [kine](https://github.com/k3s-io/kine) integration, you can run _MySQL_ or _PostgreSQL_ compatible databases as datastore for _tenant clusters_.
Kamaji offers the option of using a more capable datastore than `etcd` to save the state of multiple tenants' clusters. Thanks to the native [kine](https://github.com/k3s-io/kine) integration, you can run _MySQL_ or _PostgreSQL_ compatible databases as datastore for _Tenant Clusters_.

### Pooling
By default, Kamaji is expecting to persist all the _tenant clusters_ data in a unique datastore that could be backed by different drivers. However, you can pick a different datastore for a specific set of _tenant clusters_ that could have different resources assigned or a different tiering. Pooling of multiple datastore is an option you can leverage for a very large set of _tenant clusters_ so you can distribute the load properly. As future improvements, we have a _datastore scheduler_ feature in roadmap so that Kamaji itself can assign automatically a _tenant cluster_ to the best datastore in the pool.
By default, Kamaji is expecting to persist all the _Tenant Clusters_ data in a unique datastore that could be backed by different drivers. However, you can pick a different datastore for a specific set of _Tenant Clusters_ that could have different resources assigned or a different tiering. Pooling of multiple datastore is an option you can leverage for a very large set of _Tenant Clusters_ so you can distribute the load properly. As future improvements, we have a _datastore scheduler_ feature in roadmap so that Kamaji itself can assign automatically a _Tenant Cluster_ to the best datastore in the pool.

### Migration
In order to simplify Day2 Operations and reduce the operational burden, Kamaji provides the capability to live migrate data from a datastore to another one of the same driver without manual and error prone backup and restore operations.
Expand Down
Loading

0 comments on commit bac5d56

Please sign in to comment.