Merge branch 'master' into let-watches-be-stale

Attempt to let the tests pass

CHANGELOG.md

@@ -1,5 +1,25 @@
 ## 0.7.0 (UNRELEASED)
 
+FEATURES:
+
+* Added a new `/v1/txn` state store transaction API that allows for atomic
+  updates to and fetches from multiple entries in the key/value store. [GH-2028]
+* Script checks now support an optional `timeout` parameter. [GH-1762]
+* Reap time for failed nodes is now configurable via new `reconnect_timeout` and
+  `reconnect_timeout_wan` config options ([use with caution](https://www.consul.io/docs/agent/options.html#reconnect_timeout)). [GH-1935]
+* Consul agents can now limit the number of UDP answers returned via the DNS
+  interface. The default number of UDP answers is `3`, however by adjusting
+  the `dns_config.udp_answer_limit` configuration parameter, it is now
+  possible to limit the results down to `1`.  This tunable provides
+  environments where RFC3484 section 6, rule 9 is enforced with an important
+  workaround in order to preserve the desired behavior of randomized DNS
+  results.  Most modern environments will not need to adjust this setting as
+  this RFC was made obsolete by RFC 6724.  See the
+  [agent options](https://www.consul.io/docs/agent/options.html#udp_answer_limit)
+  documentation for additional details for when this should be
+  used. [GH-1712]
+* Prepared queries support baking in the `Near` sorting parameter [GH-2137]
+
 BACKWARDS INCOMPATIBILITIES:
 
 * `skip_leave_on_interrupt`'s default behavior is now dependent on whether or
@@ -9,6 +29,15 @@ BACKWARDS INCOMPATIBILITIES:
 
 IMPROVEMENTS:
 
+* Consul will now retry RPC calls that result in "no leader" errors for up to
+  5 seconds. This allows agents to ride out leader elections with a delayed
+  response vs. an error. [GH-2175]
+* Implemented a new set of feedback controls for the gossip layer that help
+  prevent degraded nodes that can't meet the soft real-time requirements from
+  erroneously causing `serfHealth` flapping in other, healthy nodes. [GH-2101]
+* Joins based on a DNS lookup will use TCP and attempt to join with the full
+  list of returned addresses. [GH-2101]
+* Added a new network tomogroaphy visualization to the UI. [GH-2046]
 * Consul agents will now periodically reconnect to available Consul servers
   in order to redistribute their RPC query load.  Consul clients will, by
   default, attempt to establish a new connection every 120s to 180s unless
@@ -18,31 +47,23 @@ IMPROVEMENTS:
   second per Consul server as a result of rebalancing).  Clusters in stable
   environments who use `allow_stale` should see a more even distribution of
   query load across all of their Consul servers. [GH-1743]
-* Consul agents can now limit the number of UDP answers returned via the DNS
-  interface. The default number of UDP answers is `3`, however by adjusting
-  the `dns_config.udp_answer_limit` configuration parameter, it is now
-  possible to limit the results down to `1`.  This tunable provides
-  environments where RFC3484 section 6, rule 9 is enforced with an important
-  workaround in order to preserve the desired behavior of randomized DNS
-  results.  Most modern environments will not need to adjust this setting as
-  this RFC was made obsolete by RFC 6724.  See the
-  [agent options](https://www.consul.io/docs/agent/options.html#udp_answer_limit)
-  documentation for additional details for when this should be
-  used. [GH-1712]
 * Consul will now refuse to start with a helpful message if the same UNIX
   socket is used for more than one listening endpoint. [GH-1910]
 * Removed an obsolete warning message when Consul starts on Windows. [GH-1920]
 * Defaults bind address to 127.0.0.1 when running in `-dev` mode. [GH-1878]
 * Builds Consul releases with Go 1.6.1. [GH-1948]
 * HTTP health checks limit saved output to 4K to avoid performance issues. [GH-1952]
-* Reap time for failed nodes is now configurable via new `reconnect_timeout` and
-  `reconnect_timeout_wan` config options ([use with caution](https://www.consul.io/docs/agent/options.html#reconnect_timeout)). [GH-1935]
-* Script checks now support an optional `timeout` parameter. [GH-1762]
 
 BUG FIXES:
 
+* Fixed a deadlock releated to sorting the list of available datacenters by round trip
+  time. [GH-2130]
+* Fixed an issue with the state store's immutable radix tree that would prevent it
+  from using cached modified objects during transactions, leading to extra copies
+  and increased memory / GC pressure. [GH-2106]
 * Fixed an issue where a health check's output never updates if the check
   status doesn't change after the Consul agent starts. [GH-1934]
+* External services can now be registered with ACL tokens. [GH-1738]
 
 ## 0.6.4 (March 16, 2016)
 

README.md

@@ -30,7 +30,7 @@ Consul servers only on Linux, however.
 
 ## Quick Start
 
-An extensive quick quick start is viewable on the Consul website:
+An extensive quick start is viewable on the Consul website:
 
 https://www.consul.io/intro/getting-started/install.html
 

api/kv.go

@@ -23,6 +23,43 @@ type KVPair struct {
 // KVPairs is a list of KVPair objects
 type KVPairs []*KVPair
 
+// KVOp constants give possible operations available in a KVTxn.
+type KVOp string
+
+const (
+	KVSet          KVOp = "set"
+	KVDelete            = "delete"
+	KVDeleteCAS         = "delete-cas"
+	KVDeleteTree        = "delete-tree"
+	KVCAS               = "cas"
+	KVLock              = "lock"
+	KVUnlock            = "unlock"
+	KVGet               = "get"
+	KVGetTree           = "get-tree"
+	KVCheckSession      = "check-session"
+	KVCheckIndex        = "check-index"
+)
+
+// KVTxnOp defines a single operation inside a transaction.
+type KVTxnOp struct {
+	Verb    string
+	Key     string
+	Value   []byte
+	Flags   uint64
+	Index   uint64
+	Session string
+}
+
+// KVTxnOps defines a set of operations to be performed inside a single
+// transaction.
+type KVTxnOps []*KVTxnOp
+
+// KVTxnResponse has the outcome of a transaction.
+type KVTxnResponse struct {
+	Results []*KVPair
+	Errors  TxnErrors
+}
+
 // KV is used to manipulate the K/V API
 type KV struct {
 	c *Client
@@ -238,3 +275,122 @@ func (k *KV) deleteInternal(key string, params map[string]string, q *WriteOption
 	res := strings.Contains(string(buf.Bytes()), "true")
 	return res, qm, nil
 }
+
+// TxnOp is the internal format we send to Consul. It's not specific to KV,
+// though currently only KV operations are supported.
+type TxnOp struct {
+	KV *KVTxnOp
+}
+
+// TxnOps is a list of transaction operations.
+type TxnOps []*TxnOp
+
+// TxnResult is the internal format we receive from Consul.
+type TxnResult struct {
+	KV *KVPair
+}
+
+// TxnResults is a list of TxnResult objects.
+type TxnResults []*TxnResult
+
+// TxnError is used to return information about an operation in a transaction.
+type TxnError struct {
+	OpIndex int
+	What    string
+}
+
+// TxnErrors is a list of TxnError objects.
+type TxnErrors []*TxnError
+
+// TxnResponse is the internal format we receive from Consul.
+type TxnResponse struct {
+	Results TxnResults
+	Errors  TxnErrors
+}
+
+// Txn is used to apply multiple KV operations in a single, atomic transaction.
+//
+// Note that Go will perform the required base64 encoding on the values
+// automatically because the type is a byte slice. Transactions are defined as a
+// list of operations to perform, using the KVOp constants and KVTxnOp structure
+// to define operations. If any operation fails, none of the changes are applied
+// to the state store. Note that this hides the internal raw transaction interface
+// and munges the input and output types into KV-specific ones for ease of use.
+// If there are more non-KV operations in the future we may break out a new
+// transaction API client, but it will be easy to keep this KV-specific variant
+// supported.
+//
+// Even though this is generally a write operation, we take a QueryOptions input
+// and return a QueryMeta output. If the transaction contains only read ops, then
+// Consul will fast-path it to a different endpoint internally which supports
+// consistency controls, but not blocking. If there are write operations then
+// the request will always be routed through raft and any consistency settings
+// will be ignored.
+//
+// Here's an example:
+//
+// ops := KVTxnOps{
+//     &KVTxnOp{
+//         Verb:    KVLock,
+//         Key:     "test/lock",
+//         Session: "adf4238a-882b-9ddc-4a9d-5b6758e4159e",
+//         Value:   []byte("hello"),
+//     },
+//     &KVTxnOp{
+//         Verb:    KVGet,
+//         Key:     "another/key",
+//     },
+// }
+// ok, response, _, err := kv.Txn(&ops, nil)
+//
+// If there is a problem making the transaction request then an error will be
+// returned. Otherwise, the ok value will be true if the transaction succeeded
+// or false if it was rolled back. The response is a structured return value which
+// will have the outcome of the transaction. Its Results member will have entries
+// for each operation. Deleted keys will have a nil entry in the, and to save
+// space, the Value of each key in the Results will be nil unless the operation
+// is a KVGet. If the transaction was rolled back, the Errors member will have
+// entries referencing the index of the operation that failed along with an error
+// message.
+func (k *KV) Txn(txn KVTxnOps, q *QueryOptions) (bool, *KVTxnResponse, *QueryMeta, error) {
+	r := k.c.newRequest("PUT", "/v1/txn")
+	r.setQueryOptions(q)
+
+	// Convert into the internal format since this is an all-KV txn.
+	ops := make(TxnOps, 0, len(txn))
+	for _, kvOp := range txn {
+		ops = append(ops, &TxnOp{KV: kvOp})
+	}
+	r.obj = ops
+	rtt, resp, err := k.c.doRequest(r)
+	if err != nil {
+		return false, nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	if resp.StatusCode == http.StatusOK || resp.StatusCode == http.StatusConflict {
+		var txnResp TxnResponse
+		if err := decodeBody(resp, &txnResp); err != nil {
+			return false, nil, nil, err
+		}
+
+		// Convert from the internal format.
+		kvResp := KVTxnResponse{
+			Errors: txnResp.Errors,
+		}
+		for _, result := range txnResp.Results {
+			kvResp.Results = append(kvResp.Results, result.KV)
+		}
+		return resp.StatusCode == http.StatusOK, &kvResp, qm, nil
+	}
+
+	var buf bytes.Buffer
+	if _, err := io.Copy(&buf, resp.Body); err != nil {
+		return false, nil, nil, fmt.Errorf("Failed to read response: %v", err)
+	}
+	return false, nil, nil, fmt.Errorf("Failed request: %s", buf.String())
+}