"error":"current value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)\nmax value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)"

[fsz65]

hello,when i install and run M3 by the follow steps(my env is docker,the host mechine is vmware Virtual Machine on win10):
1、docker pull quay.io/m3db/m3dbnode:latest
2、docker run -p 7201:7201 -p 7203:7203 -p 9003:9003 --name m3db -v $(pwd)/m3db_data:/var/lib/m3db quay.io/m3db/m3dbnode:latest

but i get the error message as below:

{"level":"warn","ts":1562763038.0886946,"msg":"invalid configuration found, refer to linked documentation for more information","url":"https://m3db.github.io/m3/operational_guide/kernel_configuration","error":"current value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)\nmax value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)","errorCauses":[{"error":"max value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)"},{"error":"current value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)"}]}

i used the method(https://m3db.github.io/m3/operational_guide/kernel_configuration/),and add params into /etc/sysctl.conf, but it doesnot work. the error message still......

does any body know how to solve it? thx~~~

[cirego]

I'm running into the same issue. When I startup m3db (revision d75fc9d7), I get the following message every 10 seconds:

{"level":"warn","ts":1563469022.2556548,"msg":"invalid configuration found, refer to linked documentation for more information","url":"https://m3db.github.io/m3/operational_guide/kernel_configuration","error":"current value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)\nmax value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)","errorCauses":[{"error":"max value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)"},{"error":"current value for RLIMIT_NOFILE(1048576) is below recommended threshold(3000000)"}]}

When I exec into the container, I can see that the sysctl values are set correctly:

/ # sysctl -n vm.max_map_count
3000000
/ # sysctl -n vm.swappiness
1
/ # sysctl -n fs.file-max
3000000
/ # sysctl -n fs.nr_open
3000000

Adding the --privileged flag appears to resolve the issue. Setting --cap-add=SYS_RESOURCE also resolves the issue.

EDIT: Doing a little more debugging and reading of the source code, this is the log message that I'm getting before the warnings:

{"level":"warn","ts":1563470473.3863091,"msg":"unable to raise rlimit to no file fds limit","error":"unable to raise nofile limits: rlimit_set_err=operation not permitted"}

I wrote a test program print the values from /proc/sys and syscall.Getrlimit. Here's what it returned:

SysProc: 3000000 Getrlimit: {1024 524288}

I'm going to use the --cap-add=SYS_RESOURCE workaround for the time being.

Final edit: I was able to fix this going forward using the instructions located in this Unix Stack Exchange Ulimit Question. I ended up creating a limits.conf file within the /etc/systemd/system.conf.d directory (had to make that directory too) and then rebooting.