Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs on how to deploy tidb cluster with tidb-operator in minikube #451

Merged
merged 2 commits into from
May 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ Choose one of the following tutorials:

* [Deploy TiDB by launching an AWS EKS cluster](./docs/aws-eks-tutorial.md)

* [Deploy TiDB in the minikube cluster](./docs/minikube-tutorial.md)

## User guide

Read the [user guide](./docs/user-guide.md).
Expand Down
201 changes: 201 additions & 0 deletions docs/minikube-tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Deploy TiDB in the minikube cluster

This document describes how to deploy a TiDB cluster in the [minikube](https://kubernetes.io/docs/setup/minikube/) cluster.

## Table of Contents

- [Start a Kubernetes cluster with minikube](#start-a-kubernetes-cluster-with-minikube)
* [What is minikube?](#what-is-minikube)
* [Install minikube and start a Kubernetes cluster](#install-minikube-and-start-a-kubernetes-cluster)
* [Install kubectl to access the cluster](#install-kubectl-to-access-the-cluster)
- [Install TiDB operator and run a TiDB cluster with it](#install-tidb-operator-and-run-a-tidb-cluster-with-it)
* [Install helm](#install-helm)
* [Install TiDB operator in the Kubernetes cluster](#install-tidb-operator-in-the-kubernetes-cluster)
* [Launch a TiDB cluster](#launch-a-tidb-cluster)
* [Test TiDB cluster](#test-tidb-cluster)
* [Delete TiDB cluster](#delete-tidb-cluster)
- [FAQs](#faqs)
* [TiDB cluster in minikube is not responding or responds slow](#tidb-cluster-in-minikube-is-not-responding-or-responds-slow)

## Start a Kubernetes cluster with minikube

### What is minikube?

[Minikube](https://kubernetes.io/docs/setup/minikube/) can start a local
Kubernetes cluster inside a VM on your laptop. It works on macOS, Linux, and
Windows.

### Install minikube and start a Kubernetes cluster

See [Installing Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) to install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add minimal version requirement of minikube.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

minikube (1.0.0+) on your machine.

After you installed minikube, you can run the following command to start a
Kubernetes cluster.

```
minikube start
```

For Chinese mainland users, you may use local gcr.io mirrors such as
`registry.cn-hangzhou.aliyuncs.com/google_containers`.

```
minikube start --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers
```

or configure HTTP/HTTPS proxy environments in your docker, e.g.

```
# change 127.0.0.1:1086 to your http/https proxy server IP:PORT
minikube start --docker-env https_proxy=http://127.0.0.1:1086 \
--docker-env http_proxy=http://127.0.0.1:1086
```

See [minikube setup](https://kubernetes.io/docs/setup/minikube/) for more options to
configure your virtual machine and Kubernetes cluster.

### Install kubectl to access the cluster

The Kubernetes command-line tool,
[kubectl](https://kubernetes.io/docs/user-guide/kubectl/), allows you to run
commands against Kubernetes clusters.

Install kubectl according to the instructions in [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).

After kubectl is installed, test your minikube Kubernetes cluster:

```
kubectl cluster-info
```

## Install TiDB operator and run a TiDB cluster with it

### Install helm

Helm is the package manager for Kubernetes and is what allows us to install all of the distributed components of TiDB in a single step. Helm requires both a server-side and a client-side component to be installed.

```
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get | bash
```

Install helm tiller:

```
helm init
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the default service account of tiller have the right permission?

Copy link
Contributor Author

@cofyc cofyc May 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the tiller is deployed in kube-system namespace, default account of kube-system namespace in minikube has the correct permissions required by helm.

```

If you have limited access to gcr.io, you can try a mirror, e.g.

```
helm init --upgrade --tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:$(helm version --client --short | grep -P -o 'v\d+\.\d+\.\d')
```

Once it is installed, running `helm version` should show you both the client
and server version, e.g.

```
$ helm version
Client: &version.Version{SemVer:"v2.13.1",
GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1",
GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
```

If it shows only the client version, `helm` cannot yet connect to the server. Use
`kubectl` to see if any tiller pods are running.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the related command here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, kubectl -n kube-system get pods -l app=helm is added here.


```
kubectl -n kube-system get pods -l app=helm
```

### Install TiDB operator in the Kubernetes cluster

Clone tidb-operator repository:

```
git clone https://github.com/pingcap/tidb-operator
cd tidb-operator
kubectl apply -f ./manifests/crd.yaml
helm install charts/tidb-operator --name tidb-operator --namespace tidb-admin
```

Now, we can watch the operator come up with:

```
watch kubectl get pods --namespace tidb-admin -o wide
```

If you have limited access to gcr.io (pods failed with ErrImagePull), you can
try a mirror of kube-scheduler image. You can upgrade tidb-operator like this:

```
helm upgrade tidb-operator charts/tidb-operator --namespace tidb-admin --set \
scheduler.kubeSchedulerImageName=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler
```

When you see both tidb-scheduler and tidb-controller-manager are running, you
can process to launch a TiDB cluster!

### Launch a TiDB cluster

```
helm install charts/tidb-cluster --name tidb-cluster --set \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be easier for the user to make helm changes if we have a minikube-values.yaml file. So just one option --values minikube-values.yaml

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting values from command line makes it easier for users to copy/paste. Besides, if use a separate values.yaml, we have to keep it in sync with the default values.yaml.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minikube-values.yaml would have just the values to override, it wouldn't contain all the values.

schedulerName=default-scheduler,pd.storageClassName=standard,tikv.storageClassName=standard,pd.replicas=1,tikv.replicas=1,tidb.replicas=1
```

Watch tidb-cluster up and running:

```
watch kubectl get pods --namespace default -l app.kubernetes.io/instance=tidb-cluster -o wide
```

### Test TiDB cluster

There can be a small delay between the pod is up and running, and the service
is available. You can watch list services available with:

```
watch kubectl get svc
```

When you see `tidb-cluster-tidb` appear, it's ready to connect to TiDB server.

After first, forward local port to tidb port:

```
kubectl port-forward svc/tidb-cluster-tidb 4000:4000
```

In another terminal, connect to TiDB server with MySQL client:

```
mysql -h 127.0.0.1 -P 4000 -uroot
```

Or run SQL command directly:

```
mysql -h 127.0.0.1 -P 4000 -uroot -e 'select tidb_version();'
```

### Delete TiDB cluster

```
helm delete --purge tidb-cluster
kubectl get pv -l app.kubernetes.io/instance=tidb-cluster -o name | xargs -I {} kubectl patch {} -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}} # update reclaim policy of PVs used by tidb-cluster to Delete
kubectl delete pvc -l app.kubernetes.io/managed-by=tidb-operator # delete PVCs
```

## FAQs

### TiDB cluster in minikube is not responding or responds slow

The minikube VM is configured by default to only use 2048MB of memory and 2
CPUs. You can pass more during `minikube start` with the `--memory` and `--cpus` flag.
Note that you'll need to recreate minikube VM for this to take effect.

```
minikube delete
minikube start --cpus 4 --memory 4096 ...
```