Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

Add Consul KV store based on atomic transactions #1276

Merged
merged 7 commits into from
Jan 22, 2021

Conversation

timvaillancourt
Copy link
Contributor

@timvaillancourt timvaillancourt commented Nov 30, 2020

Related issue: #1273

Description

This PR introduces a new, optional KV store for performing atomic Consul KV updates. This KV store hopes to address a race condition when performing updates of several key-values, context: #1273

This new KV store updates many KVPairs by:

  1. Getting the current values from Consul in a single transaction
  2. Skipping KVPairs that are already up-to-date (same as go/kv/consul.go)
  3. Skipping KVPairs in the local cache (same as go/kv/consul.go)
  4. Updating all KVPairs that require an update in a single transaction

A new Consul library was required to add this support, the official consul api library. Unfortunately this new library has a tonne of dependencies. At first govendor fetch github.com/hashicorp/consul/api added 1500+ files to the vendor/ dir. After manually removing various dependencies I was able to get it compile with just 500~ additional files (👎)

It seems the bulk of the dependencies come from github.com/hashicorp/go-hclog, which requires github.com/mattn/go-isatty which requires the huge library golang.org/x/sys/unix (about 300 of the 500 files) 😕. We could shave off a few more files by deleting _test.go files, but I suspect this might confuse govendor. Any ideas to reduce deps further appreciated!

Lastly, I ran govendor remove +unused to drop more deps, which explains the few unrelated files deleted from vendor/

cc @shlomi-noach

@shlomi-noach shlomi-noach marked this pull request as ready for review January 22, 2021 10:31
Copy link
Collaborator

@shlomi-noach shlomi-noach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! This being opt-in and backwards compatible is great. Thank you for adding a documentation section.

@shlomi-noach
Copy link
Collaborator

Closes #1273

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants