Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consul is duplicating the open files with reload #4121

Closed
ab-fuze opened this issue May 14, 2018 · 3 comments
Closed

Consul is duplicating the open files with reload #4121

ab-fuze opened this issue May 14, 2018 · 3 comments

Comments

@ab-fuze
Copy link

ab-fuze commented May 14, 2018

I was having issue with max open file error on ubuntu server 14.04 with consul consuming all the file descriptors.
I find out that on every consul reload the number of open files almost doubles this with 1.0.7 and 1.1.0

Reproduction Steps

To reproduce:
on one shell run
watch "lsof | grep -c consul"
on the next shell run
consul reload

Consul info for both Client and Server

Client info
consul info
agent:
	check_monitors = 7
	check_ttls = 0
	checks = 7
	services = 1
build:
	prerelease =
	revision = 5174058f
	version = 1.1.0
consul:
	known_servers = 3
	server = false
runtime:
	arch = amd64
	cpu_count = 2
	goroutines = 120
	max_procs = 2
	os = linux
	version = go1.10.1
serf_lan:
	coordinate_resets = 0
	encrypted = false
	event_queue = 0
	event_time = 1559
	failed = 0
	health_score = 0
	intent_queue = 0
	left = 0
	member_time = 53687
	members = 18
	query_queue = 0
	query_time = 49
Server info
agent:
	check_monitors = 0
	check_ttls = 4
	checks = 4
	services = 4
build:
	prerelease =
	revision = 9a494b5f
	version = 1.0.6
consul:
	bootstrap = false
	known_datacenters = 2
	leader = false
	leader_addr = 10.101.208.34:8300
	server = true
raft:
	applied_index = 47565458
	commit_index = 47565458
	fsm_pending = 0
	last_contact = 29.875453ms
	last_log_index = 47565458
	last_log_term = 200
	last_snapshot_index = 47563471
	last_snapshot_term = 200
	latest_configuration = [{Suffrage:Voter ID:468bff20-7101-7cfe-e251-19e7eec4bee9 Address:10.101.208.32:8300} {Suffrage:Voter ID:9889dbd3-012a-fa00-796e-faef203ec0c4 Address:10.101.208.34:8300} {Suffrage:Voter ID:62e4e1d2-2c4a-32b9-6b09-874952b082b1 Address:10.101.208.35:8300}]
	latest_configuration_index = 46759213
	num_peers = 2
	protocol_version = 3
	protocol_version_max = 3
	protocol_version_min = 0
	snapshot_version_max = 1
	snapshot_version_min = 0
	state = Follower
	term = 200
runtime:
	arch = amd64
	cpu_count = 8
	goroutines = 727
	max_procs = 8
	os = linux
	version = go1.9.3
serf_lan:
	coordinate_resets = 0
	encrypted = false
	event_queue = 0
	event_time = 1559
	failed = 1
	health_score = 0
	intent_queue = 0
	left = 0
	member_time = 53687
	members = 19
	query_queue = 0
	query_time = 49
serf_wan:
	coordinate_resets = 0
	encrypted = false
	event_queue = 0
	event_time = 1
	failed = 0
	health_score = 0
	intent_queue = 0
	left = 0
	member_time = 167
	members = 6
	query_queue = 0
	query_time = 1

Operating system and Environment details

Ubuntu server 14.04 on AWS EC2 instance

@ab-fuze
Copy link
Author

ab-fuze commented May 14, 2018

Just to let you know it's happeining with
1.0.6 - 1.0.7 - 1.1.0
I didn't test older versions

@guidoiaquinti
Copy link
Contributor

Duplicate of #3018, #4010 and #4093?

@ab-fuze
Copy link
Author

ab-fuze commented May 14, 2018

yes seems a duplicate of. We can link / close this I will follow the others

@ab-fuze ab-fuze closed this as completed May 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants