Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle consul_acl connection refused as a retry-able error #336

Merged
merged 4 commits into from
Nov 23, 2017
Merged

Handle consul_acl connection refused as a retry-able error #336

merged 4 commits into from
Nov 23, 2017

Conversation

kpaulisse
Copy link
Contributor

The current logic in the consul_acl provider handles and retries non-200 HTTP status codes. However "connection refused" throws its own exception which was not handled, and would cause our Puppet runs to fail when consul restarts.

Prior to this change:

Notice: /Stage[main]/Consul::Run_service/Service[consul]: Triggered 'refresh' from 1 events
Debug: /Stage[main]/Consul::Run_service/Service[consul]: The container Class[Consul::Run_service] will propagate my refresh event
Debug: Prefetching default resources for consul_acl
Debug: prefetching for anonymous
Error: Could not prefetch consul_acl provider 'default': Connection refused - connect(2) for "localhost" port 8500
Notice: /Stage[main]/Consul/Consul_acl[vault]/ensure: created
Error: /Stage[main]/Consul/Consul_acl[vault]: Could not evaluate: Connection refused - connect(2) for "localhost" port 8500
Notice: /Stage[main]/Consul/Consul_acl[anonymous]/ensure: created
Error: /Stage[main]/Consul/Consul_acl[anonymous]: Could not evaluate: Connection refused - connect(2) for "localhost" port 8500
Debug: Class[Consul::Run_service]: The container Stage[main] will propagate my refresh event
Notice: /Stage[main]/Consul/Consul_acl[agent]/ensure: created
Error: /Stage[main]/Consul/Consul_acl[agent]: Could not evaluate: Connection refused - connect(2) for "localhost" port 8500

After this change:

Notice: /Stage[main]/Consul::Run_service/Service[consul]: Triggered 'refresh' from 1 events
Debug: /Stage[main]/Consul::Run_service/Service[consul]: The container Class[Consul::Run_service] will propagate my refresh event
Debug: Prefetching default resources for consul_acl
Debug: prefetching for anonymous
Debug: http://localhost:8500/v1/acl/list?token=<redacted> Errno::ECONNREFUSED Connection refused - connect(2) for "localhost" port 8500
Debug: retrying Consul API query in 2 seconds
Debug: found {:name=>"anonymous",
 :type=>:client,
 :rules=>
  {"event"=>{""=>{"policy"=>"write"}},
   "key"=>{""=>{"policy"=>"write"}, "vault-"=>{"policy"=>"deny"}},
   "keyring"=>"read",
   "query"=>{""=>{"policy"=>"write"}},
   "service"=>{""=>{"policy"=>"write"}}},
 :id=>"anonymous",
 :acl_api_token=>"XXX",
 :port=>8500,
 :hostname=>"localhost",
 :protocol=>:http,
 :api_tries=>5,
 :ensure=>:present}
...

@kpaulisse
Copy link
Contributor Author

We've been using this internally since I pushed the commits in April, but I hadn't noticed the original CI was failing. It looks like the CI suite has changed quite a bit since then, and everything passed once I merged in the latest master, so this should be ready for review.

@solarkennedy solarkennedy merged commit a87d68c into voxpupuli:master Nov 23, 2017
@kpaulisse kpaulisse deleted the kpaulisse-handle-connection-refused branch November 23, 2017 16:54
spuder pushed a commit to spuder/puppet-consul that referenced this pull request Feb 25, 2020
…ection-refused

Handle consul_acl connection refused as a retry-able error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants