Add getTxStatus RPC

CMakeLists.txt

@@ -341,6 +341,11 @@ if(BUILD_TESTS)
                             secp256k1.host http_parser.host sss.host
     )
 
+    add_unit_test(
+      tx_status_test
+      ${CMAKE_CURRENT_SOURCE_DIR}/src/node/rpc/test/tx_status_test.cpp
+    )
+
     add_unit_test(
       member_voting_test
       ${CMAKE_CURRENT_SOURCE_DIR}/src/node/rpc/test/member_voting_test.cpp

doc/schemas/tx_params.json

@@ -0,0 +1,21 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "properties": {
+    "seqno": {
+      "maximum": 4294967295,
+      "minimum": 0,
+      "type": "number"
+    },
+    "view": {
+      "maximum": 18446744073709551615,
+      "minimum": 0,
+      "type": "number"
+    }
+  },
+  "required": [
+    "view",
+    "seqno"
+  ],
+  "title": "tx/params",
+  "type": "object"
+}

doc/schemas/tx_result.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "properties": {
+    "status": {
+      "enum": [
+        "UNKNOWN",
+        "PENDING",
+        "COMMITTED",
+        "INVALID"
+      ]
+    }
+  },
+  "required": [
+    "status"
+  ],
+  "title": "tx/result",
+  "type": "object"
+}

doc/users/issue_commands.rst

@@ -52,29 +52,41 @@ Checking for Commit
 
 Because of the decentralised nature of CCF, a request is committed to the ledger only once a number of nodes have agreed on that request.
 
-To guarantee that their request is successfully committed to the ledger, a user needs to issue a ``getCommit`` request, specifying the ``commit`` version received in the response. If CCF returns a ``global-commit`` greater than the ``commit`` version at which the ``LOG_record`` request was issued `and` that result ``commit`` is in the same ``term``, then the request was committed to the ledger.
+To guarantee that their request is successfully committed to the ledger, a user should issue a ``GET /tx`` request, specifying the version received in the response. This version is constructed from a view (also called term in some places) and a sequence number (or commit index).
 
 .. code-block:: bash
 
-    $ cat get_commit.json
-    {
-      "commit": 23
-    }
-
-    $ curl https://<ccf-node-address>/users/getCommit --cacert networkcert.pem --key user0_privk.pem --cert user0_cert.pem --data-binary @get_commit.json -H "content-type: application/json" -i
+    $ curl -X GET "https://<ccf-node-address>/users/tx?view=2&seqno=18" --cacert networkcert.pem --key user0_privk.pem --cert user0_cert.pem -i
     HTTP/1.1 200 OK
-    content-length: 32
+    content-length: 23
     content-type: application/json
-    x-ccf-commit: 33
-    x-ccf-global-commit: 33
-    x-ccf-term: 2
+    x-ccf-commit: 42
+    x-ccf-global-commit: 40
+    x-ccf-term: 5
 
-    {
-      "commit": 23,
-      "term": 2
-    }
+    {"status":"COMMITTED"}
+
+This example queries the status of transaction ID 2.18 (constructed from view 2 and sequence number 18). The response indicates this was successfully committed. The headers also show that the service has since made progress with other requests, and global commit index has continued to increase.
+
+The possible statuses are:
 
-In this example, the ``result`` field indicates that the request was executed at ``23`` (``commit``), and in term ``2``, the same term that the ``LOG_record`` executed in. Moreover, the ``global_commit`` (``33``) is now greater than the ``commit`` version. The ``LOG_record`` request issued earlier was successfully committed to the ledger.
+- ``UNKNOWN`` - this node has not received a transaction with the given ID
+- ``PENDING`` - this node has received a transaction with the given ID, but does not yet know if the transaction has been committed
+- ``COMMITTED`` - this node knows that this transaction is committed, it is an irrevocable and durable part of the service's transaction history
+- ``INVALID`` - this node knows that the given transaction cannot be committed. This occurs when the view changes, and some pending transactions may be lost and must be resubmitted, but also applies to IDs which are known to be impossible given the current globally committed IDs
+
+On a given node, the possible transitions between states are described in the following diagram:
+
+.. mermaid::
+
+    stateDiagram
+        Unknown --> Pending
+        Pending --> Committed
+        Pending --> Invalid
+
+It is possible that intermediate states are not visible (eg - a transition from Unknown to Committed may never publically show a Pending result). Nodes may disagree on the current state due to communication delays, but will never disagree on transitions (in other words, they may believe a Committed transaction is still Unknown or Pending, but will never report it as Invalid).
+
+Note that transaction IDs are uniquely assigned by the service - once a request has been assigned an ID, this ID will never be associated with a different write transaction. In normal operation, the next requests will be given versions 2.19, then 2.20, and so on, and after a short delay 2.18 will be committed. If requests are submitted in parallel, they will be applied in a consistent order indicated by their assigned versions. If the network is unable to reach consensus, it will trigger a leadership election which increments the view. In this case the user's next request may be given a version 3.16, followed by 3.17, then 3.18. The sequence number is reused, but in a different view; the service knows that 2.18 can never be assigned, so it can report this as an invalid ID. Read-only transactions are an exception - they do not get a unique transaction ID but instead return the ID of the last write transaction whose state they may have read.
 
 Transaction receipts
 --------------------
@@ -85,12 +97,7 @@ To obtain a receipt, a user needs to issue a ``getReceipt`` RPC for a particular
 
 .. code-block:: bash
 
-    $ cat get_receipt.json
-    {
-      "commit": 23
-    }
-
-    $ curl https://<ccf-node-address>/users/getReceipt --cacert networkcert.pem --key user0_privk.pem --cert user0_cert.pem --data-binary @get_receipt.json -H "content-type: application/json"
+    $ curl -X GET "https://<ccf-node-address>/users/getReceipt?commit=23" --cacert networkcert.pem --key user0_privk.pem --cert user0_cert.pem
     {
       "receipt": [ ... ],
     }

doc/users/rpc_api.rst

@@ -65,8 +65,6 @@ Common Methods
 getCommit
 ~~~~~~~~~
 
-.. literalinclude:: ../schemas/getCommit_params.json
-    :language: json
 .. literalinclude:: ../schemas/getCommit_result.json
     :language: json
 

samples/perf_client/perf_client.h

@@ -7,7 +7,6 @@
 
 // CCF
 #include "clients/rpc_tls_client.h"
-#include "clients/sig_rpc_tls_client.h"
 #include "ds/cli_helper.h"
 #include "ds/files.h"
 #include "ds/logger.h"
@@ -304,18 +303,17 @@ namespace client
       // Create a cert if this is our first rpc_connection
       const bool is_first = get_cert();
 
-      const auto conn = (options.sign && !force_unsigned) ?
-        std::make_shared<SigRpcTlsClient>(
-          key,
-          options.server_address.hostname,
-          options.server_address.port,
-          nullptr,
-          tls_cert) :
-        std::make_shared<RpcTlsClient>(
-          options.server_address.hostname,
-          options.server_address.port,
-          nullptr,
-          tls_cert);
+      auto conn = std::make_shared<RpcTlsClient>(
+        options.server_address.hostname,
+        options.server_address.port,
+        nullptr,
+        tls_cert);
+
+      if (options.sign && !force_unsigned)
+      {
+        conn->create_key_pair(key);
+      }
+
       conn->set_prefix("users");
 
       // Report ciphersuite of first client (assume it is the same for each)

samples/perf_client/timing.h

@@ -212,35 +212,37 @@ namespace timing
       receives.push_back({Clock::now() - start_time, rpc_id, commit});
     }
 
-    // Repeatedly calls getCommit RPC until the target index has been committed
-    // (or will never be committed), checks it was in expected term, returns
-    // first confirming response. Calls received_response for each response, if
-    // record is true. Throws on errors, or if target is rolled back
+    // Repeatedly calls GET /tx RPC until the target seqno has been
+    // committed (or will never be committed), returns first confirming
+    // response. Calls record_[send/response], if record is true.
+    // Throws on errors, or if target is rolled back
     RpcTlsClient::Response wait_for_global_commit(
       const CommitPoint& target, bool record = true)
     {
       auto params = nlohmann::json::object();
-      params["commit"] = target.index;
+      params["view"] = target.term;
+      params["seqno"] = target.index;
 
-      constexpr auto get_commit = "getCommit";
+      constexpr auto get_tx_status = "tx";
 
       LOG_INFO_FMT(
-        "Waiting for global commit {}.{}", target.term, target.index);
+        "Waiting for transaction ID {}.{}", target.term, target.index);
 
       while (true)
       {
-        const auto response = net_client->call(get_commit, params);
+        const auto response = net_client->get(get_tx_status, params);
 
         if (record)
         {
-          record_send(get_commit, response.id, false);
+          record_send(get_tx_status, response.id, false);
         }
 
         const auto body = net_client->unpack_body(response);
         if (response.status != HTTP_STATUS_OK)
         {
           throw runtime_error(fmt::format(
-            "getCommit failed with status {}: {}",
+            "{} failed with status {}: {}",
+            get_tx_status,
             http_status_str(response.status),
             body.dump()));
         }
@@ -251,49 +253,36 @@ namespace timing
           record_receive(response.id, commit_ids);
         }
 
-        const auto response_term = body["term"].get<size_t>();
-        if (response_term == 0)
+        // NB: Eventual header re-org should be exposing API types so
+        // they can be consumed cleanly from C++ clients
+        const auto tx_status = body["status"];
+        if (tx_status == "PENDING" || tx_status == "UNKNOWN")
         {
           // Commit is pending, poll again
           this_thread::sleep_for(10us);
           continue;
         }
-        else if (response_term < target.term)
+        else if (tx_status == "COMMITTED")
         {
-          throw std::logic_error(fmt::format(
-            "Unexpected situation - {} was committed in old term {} (expected "
-            "{})",
-            target.index,
-            response_term,
-            target.term));
+          LOG_INFO_FMT("Found global commit {}.{}", target.term, target.index);
+          LOG_INFO_FMT(
+            " (headers term: {}, local: {}, global: {})",
+            commit_ids.term,
+            commit_ids.local,
+            commit_ids.global);
+          return response;
         }
-        else if (response_term == target.term)
+        else if (tx_status == "INVALID")
         {
-          // Good, this target commit was committed in the expected term
-          if (commit_ids.global >= target.index)
-          {
-            // ...and this commit has been globally committed
-            LOG_INFO_FMT(
-              "Found global commit {}.{}", target.term, target.index);
-            LOG_INFO_FMT(
-              " (headers term: {}, local: {}, global: {})",
-              commit_ids.term,
-              commit_ids.local,
-              commit_ids.global);
-            return response;
-          }
-
-          // else global commit is still pending
-          continue;
+          throw std::logic_error(fmt::format(
+            "Transaction {}.{} is now marked as invalid",
+            target.term,
+            target.index));
         }
         else
         {
-          throw std::logic_error(fmt::format(
-            "Pending transaction was dropped! Looking for {}.{}, but term has "
-            "advanced to {}",
-            target.term,
-            target.index,
-            response_term));
+          throw std::logic_error(
+            fmt::format("Unhandled tx status: {}", tx_status));
         }
       }
     }

sphinx/source/schemas/tx_params.json

@@ -0,0 +1,21 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "properties": {
+    "seqno": {
+      "maximum": 4294967295,
+      "minimum": 0,
+      "type": "number"
+    },
+    "view": {
+      "maximum": 18446744073709551615,
+      "minimum": 0,
+      "type": "number"
+    }
+  },
+  "required": [
+    "view",
+    "seqno"
+  ],
+  "title": "tx/params",
+  "type": "object"
+}

sphinx/source/schemas/tx_result.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "properties": {
+    "status": {
+      "enum": [
+        "UNKNOWN",
+        "PENDING",
+        "COMMITTED",
+        "INVALID"
+      ]
+    }
+  },
+  "required": [
+    "status"
+  ],
+  "title": "tx/result",
+  "type": "object"
+}