Skip to content
This repository has been archived by the owner on Feb 9, 2022. It is now read-only.

Commit

Permalink
Update troubleshooting.
Browse files Browse the repository at this point in the history
  • Loading branch information
falfaro committed Mar 14, 2019
1 parent 7e201b7 commit e7db93c
Showing 1 changed file with 17 additions and 1 deletion.
18 changes: 17 additions & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -456,7 +456,23 @@ Address: 50.17.235.25

### Elasticsearch crashloop under EKS

EKS clusters which use the AMI image named `amazon-eks-node-1.10-v20190220` in any nodegroup are known to be affected by a [bug](https://github.com/awslabs/amazon-eks-ami/issues/193) that causes Elasticsearch 6.0 to get into a crashloop. In order to fix this, you will SSH into every node from the EKS nodegroup and remove the `ulimit` configuration of the Docker daemon:
When Elasticsearch enters into a never-ending crashloop under EKS, the logs can reveal the cause:

```console
$ kubectl --namespace=kubeprod logs elasticsearch-logging-0 elasticsearch-logging
```

__Troubleshooting__:

If you see logged mesasges like this:

```
[1]: max file descriptors [8192] for elasticsearch process is too low, increase to at least [65536]
```

Then, you are affected by a [bug](https://github.com/awslabs/amazon-eks-ami/issues/193) that causes Elasticsearch 6.0 to get into a crashloop because of incorrect `ulimit` values. It affects EKS clusters which use the AMI image named `amazon-eks-node-1.10-v20190220`.

In order to fix this, you will SSH into every node from the EKS nodegroup and remove the `ulimit` configuration of the Docker daemon:

```console
sudo sed -i '/"nofile": {/,/}/d' /etc/docker/daemon.json
Expand Down

0 comments on commit e7db93c

Please sign in to comment.