From 950e8b3fd1a46fcac37e702d3c3e5181646de706 Mon Sep 17 00:00:00 2001 From: qrest Date: Tue, 5 Oct 2021 10:22:16 +0000 Subject: [PATCH] Update to upstream btcd v0.22.0-beta (#3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Improve error message about non-active segwit on simnet I started playing with simnet and was confronted with error message: ``` [ERR] FNDG: Unable to broadcast funding tx for ChannelPoint(:0): -22: TX rejected: transaction has witness data, but segwit isn't active yet ``` I wasn't aware of the activation period so I got quite puzzled. Google helped. But I think the message could mention likely cause. Newly it optionally prints something like: ``` (The threshold for segwit activation is 300 blocks on simnet, current best height is 113) ``` * btcctl: add regtest mode to btcctl * build: replace travis-ci with github actions. test go 1.14 use golangci-lint * build: update deps * build: clean linter warnings * btcjson: change getblock default verbosity to 1 This change makes btcd's getblock command match bitcoind's. Previously the default verbosity was 0, which caused errors when using the rpcclient library to connect to a bitcoind node - getblock would unmarshall incorrectly since it didn't expect a verbosity=1 result when it did not specify verbosity. * rpcclient: send legacy GetBlock request for backwards compatibility Without this, users of this library wouldn't be able to issue GetBlock requests to nodes which haven't updated to support the latest request format, namely the use of a single `int` parameter to denote verbosity instead of two `bool`s. * rpcclient: Add cookie auth Based on Hugo Landau's cookie auth implementation for Namecoin's ncdns. Fixes https://github.com/btcsuite/btcd/issues/1054 * rpcclient: Refactor cookie caching * rpcclient: Try user+pass auth before cookie auth * rpcclient: Read first line of cookie instead of trimming space * rpcclient: serialize nil inputs to empty list * Improve chain state init efficiency Remove unnecessary slice of all block indexes and remove DB iteration over all block indexes that used to determined the size of the slice. * Add blockchain.NewUtxoEntry() to directly create entries for UtxoViewpoint The current methods to add to a UtxoViewpoint don't allow for a situation where we have only UTXO data but not a whole transaction. This commit allows contstruction of a UtxoEntry without requiring a full MsgTx. AddTxOut() and AddTxOuts() both require a whole transaction, including the inputs, which are only used in order to calculate the txid. In some situations, such as with use of the utreexo accumulator, we only have the utxo data but not the transaction which created it. For reference, utreexo's initial usage of the blockchain.NewUtxoEntry() function is at https://github.com/mit-dci/utreexo/pull/135/files#diff-3f7b8f9991ea957f1f4ad9f5a95415f0R96 * Add getchaintxstats JSON-RPC client command * Add fundrawtransaction RPC call * Add getbalances RPC client command * rpcclient: Add GetTransactionWatchOnly method * peer: knownInventory, sentNonces - use generic lru While here, also rename and generalize limitMap and apply to other maps which need to be bounded. * btcec: Avoid panic in fieldVal.SetByteSlice for large inputs The implementation has been adapted from the dcrec module in dcrd. The bug was initially fixed in decred/dcrd@3d9cda1 while transitioning to a constant time algorithm. A large set of test vectors were subsequently added in decred/dcrd@8c6b52d. The function signature has been preserved for backwards compatibility. This means that returning whether the value has overflowed, and the corresponding test vectors have not been backported. This fixes #1170 and closes a previous attempt to fix the bug in #1178. * config+service_windows: add flag to disable win service To run integration tests with btcd on Windows in non-interactive environments (such as the Travis build with Windows machines), we need to make sure we can still spawn a child process instead of only a windows background service. * updated docs for getblock-verbosity fixes * Update json_rpc_api.md Corrections suggested by @onyb https://github.com/btcsuite/btcd/pull/1608#discussion_r458363077 * netsync: handle notfound messages from peers backport from https://github.com/decred/dcrd/pull/2253 When a peer sends a notfound message, remove the hash from requested map. Also increase notfound ban score and return early if it disconnects the peer. * release: update release script path * release: remove old scripts and update process doc - remove prep_release.sh and notes.sample - update license in release.sh - add notes for maintainers on the release process - mention CHANGES file modifications * Update CHANGES file for 0.21.0 release Also updated changes for 0.20.1, and added a small note about changes since 0.12.0. * btcd: bump version to v0.21.0-beta * blockchain: remove unknown block version warning * Add rpclient implementation of getdescriptorinfo RPC * peer: prevent last block height going backwards This modifies the UpdateLastBlockHeight function to ensure the new height is after the existing height before updating it in order to prevent it from going backwards so it properly matches the intent of the function which is to report the latest known block height for the peer. Without this change, the value will properly start out at the latest known block height reported by the peer during version negotiation, however, it will be set to lower values when syncing from the peer due to requesting old blocks and blindly updating the height. It also adds a test to ensure proper functionality. This is a backport of https://github.com/decred/dcrd/pull/1747 * Fix monetary unit * rpcserver: add parity with bitcoind for validateaddress Updated the rpcserver handler for validateaddress JSON-RPC command to have parity with the bitcoind 0.20.0 interface. The new fields included are - isscript, iswitness, witness_version, and witness_program. The scriptPubKey field has been left out since it requires wallet access. This update has no impact on the rpcclient.ValidateAddress method, which uses the btcjson.ValidateAddressWalletResult type for modelling the response from bitcoind. * Add getblockfilter JSON-RPC client command Add type for second getblockfilter param * Implement signmessagewithprivkey JSON-RPC command Reuse the Bitcoin message signature header const also in verifymessage. * rpcclient: Implement importmulti JSON-RPC client command * Add Dockerfile to build and run btcd on Docker. * btcd: fix conversion of int to string failing in Go 1.15 * btcjson,wire: fix invalid use of string(x) to convert byte value * Major rework on documentation to make it compatible to readthedocs.org * Added symlink to index.md for github readme preview. * btcd+netsync: support witness tx and block in notfound msg * btcec: set curve name in CurveParams Set curve name(secp256k1) in KoblitzCurve.CurveParams Fixes #1564 * btcec: add a comment indicating where curve name taken from Related with #1565 * rpcclient: support listtransactions RPC with watchonly argument Co-authored-by: Gert-Jaap Glasbergen * blockchain: Remove unnecessary tx hash * btcjson: update ListTransactionsResult for Bitcoin 0.20.0 This only adds new fields as optional, in order to make this change backwards compatible with older versions of Bitcoin Core. * chaincfg: Add RegisterHDKeyID func to populate HD key ID pairs Currently, the only way to register HD version bytes is by initializing chaincfg.Params struct, and registering it during package init. RegisterHDKeyID provides a way to populate custom HD version bytes, without having to create new chaincfg.Params instances. This is useful for library packages who want to use non-standard version bytes for serializing extended keys, such as the ones documented in SLIP-0132. This function is complementary to HDPrivateKeyToPublicKeyID, which is used to lookup previously registered key IDs. * Nullable optional JSON-RPC parameters Fix command marshalling dropping params following params with nil value. #1591 Allow specifying null parameter value from command line. * GitHub Actions: Enable Go Race detector and code coverage This modifies the goclean.sh script to run tests with the race detector enabled. It also enables code coverage, and uploads the results to coveralls.io. Running tests with -race and -cover flags was disabled in 6487ba1 and 6788df7 respectively, due to some limits on time/goroutines being hit on Travis CI. Since we have migrated to GitHub Actions, it is desirable to bring them back. * rpc: Add getnodeaddresses JSON-RPC support Add NodeAddresses function to rpcserverConnManager interface for fetching known node addresses. * btcjson,rpcclient: add support for PSBT commands to rpcclient * Added ListSinceBlockMinConfWatchOnly method. * wire: add proper types for flag field and improve docs Summary of changes: - Add a new const TxFlagMarker to indicate the flag prefix byte. - Add a new TxFlag type to enumerate the flags supported by the tx parser. This allows us to avoid hardcoded magics, and will make it easier to support new flags in future. - Improve code comments. Closes #1598. * removed unnecessary GOMAXPROCS function calls * rpcclient: add deriveaddresses RPC command * ci: add go 1.15 to tests * sample-btcd.conf: fix typo * btcjson: add test for null params in searchrawtransactions Closes PR #1476. * GetBlockTemplate RPC client implementation (#1629) * GetBlockTemplate RPC client implementation * Txid added to the getblocktemplate result * Omitempty for TxID and improved comment for GetBlockTemplate 'rules' field * rpcclient: implement getaddressinfo command Fields such as label, and labelspurpose are not included, since they are deprecated, and will be removed in Bitcoin Core 0.21. * Fix link to using bootstrap.dat * rpcclient: implement getwalletinfo command * rpcserver: add txid to getblocktemplate response * rpc: add signrawtransactionwithwallet interface Adds interface for issuing a signrawtransactionwithwallet command. Note that this does not add functionality for the btcd rpc server itself, it simply assumes that the RPC client has this ability and gives an API for interacting with the RPC client. rpc: add signrawtransactionwithwallet interface * rpcclient: implement gettxoutsetinfo command * Unmarshal hashes/second as float in GetMiningInfoResult * rpcclient: add more wallet commands Implement backupwallet, dumpwallet, loadwallet and unloadwallet. * btcjson: add new JSON-RPC errors and document them * rpcclient: implement createwallet with functional options * rpcclient: fix documentation typos * integration: allow setting custom btcd exe path To allow using a custom btcd executable, we allow specifying a path to a file. If the path is empty, the harness will fall back to compiling one from scratch. * integration: allow overwriting address generator * integration: allow specifying connection behavior * integration/rpctest: randomizes port in rpctest.New to reduce collisions * btcjson+rpcserverhelp: restore bitcoind compatibility The PR #1594 introduced a change that made the order of parameters relevant, if one of them is nil. This makes it harder to be backward compatible with the same JSON message if an existing parameter in bitcoind was re-purposed to have a different meaning. * simplify s[:] to s where s is a slice Found using https://go-critic.github.io/overview#unslice-ref * rpcclient: add ExtraHeaders in ConnConfig * Add support for receiving sendaddrv2 message from a peer * fixed broken link * Add support for arm32v7 in Dockerfile * Fixes btcsuite/btcd#1653 * btcjson: Update fields in GetBlockChainInfoResult Update the fields of GetBlockChainInfoResult to reflect the current state of the RPC returned by other full-node implementations. * InitialBlockDownload - Node is in Initial Block Download mode if True. * SizeOnDisk - The estimated size of the block and undo files on disk. * txscript: add benchmark for IsUnspendable - create benchmarks to measure allocations - add test for benchmark input - create a low alloc parseScriptTemplate - refactor parsing logic for a single opcode * txscript/hashcache_test: always add inputs during getTxn TestHashCacheAddContainsHashes flakes fairly regularly when rebasing PR #1684 with: txid wasn't inserted into cache but was found. With probabilty 1/10^2 there will be no inputs on the transaction. This reduces the entropy in the txid, and I belive is the primary cause of the flake. * txscript/hashcache_test: call rand.Seed once in init This resolves the more fundamental flake in the unit tests noted in the prior commit. Because multiple unit tests call rand.Seed in parallel, it's possible they can be executed with the same unix timestamp (in seconds). If the second call happens between generating the hash cache and checking that the cache doesn't contain a random txn, the random transaction is in fact a duplicate of one generated earlier since the RNG state was reset. To remedy, we initialize rand.Seed once in the init function. * btcec: validate R and S signature components in RecoverCompact * Add Batch JSON-RPC support (rpc client & server) * Fix error message returned by EstimateFee When you provide an argument to EstimateFee(numblocks uint32) that exceeds the estimateFeeDepth (which is set to 25), you get an error message that says "can only estimate fees for up to 100 blocks from now". The variable used in the if condition and the variable used for creating the error message should be the same. * docs: update shields * rpcserver: Fix Error message returned by processRequest When processRequest can't find a rpc command, standardCmdResult returns a `btcjson.ErrRPCMethodNotFound` but it gets ignored and a `btcjson.ErrRPCInvalidRequest` is returned instead. This makes processRequest return the right error message. * peer: allow external testing of peer.Peer The previous use of allowSelfConns prevented this, as users aren't able to invoke peer.TstAllowSelfConns themselves due to being part of a test file, which aren't exported at the library level, leading to a "disconnecting peer connected to self" error upon establishing a mock connection between two peers. By including the option at the config level instead (false by default, prevents connections to self) we enable users of the peer library to properly test the behavior of the peer.Peer struct externally. * addrmgr: Use RLock/RUnlock when possible * build: update btcutil dependency * rpcclient: fix documentation typo * btcjson: Updated TxRawResult.Version from int32 to uint32 * wire+chaincfg: add signet params This commit adds all necessary chain parameters for connecting to the public signet network. Reference: https://github.com/bitcoin/bitcoin/pull/18267 * config+params: add signet config option This commit adds the --signet command line flag (or signet config option) for starting btcd in signet mode. * rpcserver: add taproot deployment to getblockchaininfo * btcctl: add signet param This commit adds the --signet command line flag to the btcctl utility. * mining: extract witness commitment add into method * rpctest: add witness commitment when calling CreateBlock If we tried to include transactions having witnesses, the block would be invalid since the witness commitment was not added. * Don't reference the readme that we don't produce * chaincfg: fix deployment bit numbers On signet all previous soft forks and also taproot are always activated, meaning the version is always 0x20000000 for all blocks. To make sure they activate properly in `btcd` we therefore need to use the correct bit to mask the version. This means that on any custom signet there would need to be 2016 blocks mined before SegWit or Taproot can be used. * mempool: add additional test case for inherited RBF replacement In this commit, we add an additional test case for inherited RBF replacement. This test case asserts that if a parent is marked as being replaceable, but the child isn't, then the child can still be replaced as according to BIP 125 it shoudl _inhreit_ the replaceability of its parent. The addition of this test case was prompted by the recently discovered Bitcoin Core "CVE" [1]. It turns out that bitcoind doesn't properly implement BIP 125. Namely it fails to allow a child to "inherit" replaceability if its parent is also replaceable. Our implementation makes this trait rather explicit due to its recursive implementation. Kudos to the original implementer @wpaulino for getting this correct. [1]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-May/018893.html. * Update CHANGES file for 0.22.0 release * btcd: bump version to v0.22.0-beta * Update release date for v0.22.0-beta in CHANGES file * remove duplicate command Co-authored-by: Antonin Hildebrand Co-authored-by: Dan Cline Co-authored-by: David Hill Co-authored-by: Henry Co-authored-by: Wilmer Paulino Co-authored-by: Olaoluwa Osuntokun Co-authored-by: JeremyRand Co-authored-by: Torkel Rogstad Co-authored-by: Mikael Lindlof Co-authored-by: adiabat Co-authored-by: Federico Bond Co-authored-by: Anirudha Bose Co-authored-by: Javed Khan Co-authored-by: Anirudha Bose Co-authored-by: Oliver Gugger Co-authored-by: qqjettkgjzhxmwj <37233887+JettScythe@users.noreply.github.com> Co-authored-by: Dan Cline Co-authored-by: John C. Vernaleo Co-authored-by: wakiyamap Co-authored-by: Christian Lehmann Co-authored-by: yyforyongyu Co-authored-by: Hanjun Kim Co-authored-by: Gert-Jaap Glasbergen Co-authored-by: Calvin Kim Co-authored-by: Andrew Tugarinov Co-authored-by: ipriver Co-authored-by: Jake Sylvestre Co-authored-by: Tristyn Co-authored-by: Elliott Minns Co-authored-by: Friedger Müffke Co-authored-by: David Mazary Co-authored-by: Armando Ochoa Co-authored-by: Liran Sharir Co-authored-by: Iskander Sharipov Co-authored-by: 10gic <2391796+10gic@users.noreply.github.com> Co-authored-by: Yaacov Akiba Slama Co-authored-by: ebiiim Co-authored-by: Victor Lavaud Co-authored-by: Vinayak Borkar Co-authored-by: Steven Kreuzer Co-authored-by: Conner Fromknecht Co-authored-by: Appelberg-s Co-authored-by: Gustavo Chain Co-authored-by: Aurèle Oulès Co-authored-by: Johan T. Halseth --- .github/workflows/go.yml | 7 +- .gitignore | 4 + CHANGES | 122 +++- Dockerfile | 41 ++ README.md | 17 +- addrmgr/addrmanager.go | 14 +- blockchain/README.md | 10 +- blockchain/chain.go | 15 +- blockchain/chainio.go | 4 +- blockchain/fullblocktests/README.md | 4 +- blockchain/indexers/README.md | 4 +- blockchain/scriptval_test.go | 3 - blockchain/thresholdstate.go | 10 +- blockchain/validate.go | 3 +- blockchain/versionbits.go | 44 -- btcd.go | 3 - btcec/README.md | 12 +- btcec/btcec.go | 2 + btcec/signature.go | 21 +- btcec/signature_test.go | 34 ++ btcjson/README.md | 12 +- btcjson/btcdextcmds_test.go | 4 +- btcjson/btcwalletextcmds_test.go | 4 +- btcjson/chainsvrcmds.go | 235 ++++++-- btcjson/chainsvrcmds_test.go | 180 +++++- btcjson/chainsvrresults.go | 120 +++- btcjson/chainsvrresults_test.go | 113 ++++ btcjson/chainsvrwscmds_test.go | 4 +- btcjson/chainsvrwsntfns_test.go | 4 +- btcjson/cmdparse.go | 20 +- btcjson/cmdparse_test.go | 86 ++- btcjson/example_test.go | 8 +- btcjson/helpers.go | 8 + btcjson/jsonrpc.go | 150 ++++- btcjson/jsonrpc_test.go | 12 +- btcjson/jsonrpcerr.go | 133 ++++- btcjson/walletsvrcmds.go | 404 ++++++++++++- btcjson/walletsvrcmds_test.go | 557 +++++++++++++++++- btcjson/walletsvrresults.go | 213 ++++++- btcjson/walletsvrresults_test.go | 127 ++++ btcjson/walletsvrwscmds_test.go | 4 +- btcjson/walletsvrwsntfns_test.go | 4 +- chaincfg/README.md | 4 +- chaincfg/chainhash/README.md | 4 +- chaincfg/genesis.go | 28 + chaincfg/genesis_test.go | 68 +++ chaincfg/params.go | 165 +++++- chaincfg/params_test.go | 102 +++- cmd/addblock/addblock.go | 4 +- cmd/btcctl/btcctl.go | 2 +- cmd/btcctl/config.go | 11 + cmd/btcctl/version.go | 2 +- config.go | 52 +- connmgr/README.md | 4 +- database/README.md | 8 +- database/cmd/dbtool/main.go | 4 - database/ffldb/README.md | 4 +- database/ffldb/blockio.go | 2 +- database/ffldb/driver_test.go | 2 - database/internal/treap/README.md | 4 +- doc.go | 2 +- docs/.gitignore | 1 + docs/Makefile | 20 + docs/README.md | 297 +--------- docs/code_contribution_guidelines.md | 108 ++-- docs/conf.py | 77 +++ docs/configuration.md | 190 ++++++ ...configure_peer_server_listen_interfaces.md | 43 -- .../configure_rpc_server_listen_interfaces.md | 47 -- docs/configuring_tor.md | 96 +-- docs/contact.md | 15 + docs/controlling.md | 34 ++ docs/default_ports.md | 15 - docs/developer_resources.md | 37 ++ docs/index.md | 57 ++ docs/installation.md | 76 +++ docs/json_rpc_api.md | 3 +- docs/make.bat | 35 ++ docs/mining.md | 30 + docs/requirements.txt | 1 + docs/table_of_content.md | 13 + docs/update.md | 8 + docs/using_bootstrap_dat.md | 79 --- docs/using_docker.md | 160 +++++ docs/wallet.md | 5 + go.mod | 6 +- go.sum | 5 +- goclean.sh | 5 +- integration/README.md | 2 +- integration/bip0009_test.go | 18 +- integration/csv_fork_test.go | 40 +- integration/rpcserver_test.go | 64 +- integration/rpctest/README.md | 4 +- integration/rpctest/blockgen.go | 16 + integration/rpctest/memwallet.go | 6 +- integration/rpctest/node.go | 16 +- integration/rpctest/rpc_harness.go | 131 ++-- integration/rpctest/rpc_harness_test.go | 30 +- integration/rpctest/utils.go | 14 +- mempool/README.md | 4 +- mempool/estimatefee.go | 2 +- mempool/mempool_test.go | 41 ++ mining/README.md | 4 +- mining/cpuminer/README.md | 4 +- mining/mining.go | 78 +-- netsync/README.md | 4 +- netsync/manager.go | 5 + params.go | 7 + peer/README.md | 6 +- peer/doc.go | 2 +- peer/example_test.go | 2 + peer/export_test.go | 18 - peer/peer.go | 16 +- peer/peer_test.go | 74 ++- release/README.md | 11 +- release/release.sh | 1 - rpcadapters.go | 9 + rpcclient/README.md | 6 +- rpcclient/chain.go | 158 +++++ rpcclient/example_test.go | 156 +++++ .../examples/bitcoincorehttpbulk/README.md | 31 + .../examples/bitcoincorehttpbulk/main.go | 46 ++ rpcclient/infrastructure.go | 176 +++++- rpcclient/mining.go | 37 +- rpcclient/net.go | 39 +- rpcclient/rawrequest.go | 2 +- rpcclient/rawtransactions.go | 145 ++++- rpcclient/wallet.go | 476 ++++++++++++++- rpcserver.go | 541 +++++++++++++---- rpcserverhelp.go | 133 +++-- rpcwebsocket.go | 554 ++++++++++++----- sample-btcd.conf | 2 +- server.go | 4 + txscript/README.md | 10 +- txscript/hashcache_test.go | 14 +- txscript/opcode.go | 73 +++ txscript/script.go | 114 ++-- txscript/script_test.go | 35 ++ txscript/standard.go | 20 +- txscript/standard_test.go | 40 +- upnp.go | 3 +- version.go | 2 +- wire/README.md | 4 +- wire/fixedIO_test.go | 2 +- wire/message.go | 8 +- wire/msgsendaddrv2.go | 42 ++ wire/msgtx.go | 70 ++- wire/msgtx_test.go | 12 +- 148 files changed, 6710 insertions(+), 1585 deletions(-) create mode 100644 Dockerfile create mode 100644 btcjson/walletsvrresults_test.go create mode 100644 docs/.gitignore create mode 100644 docs/Makefile mode change 100644 => 120000 docs/README.md create mode 100644 docs/conf.py create mode 100644 docs/configuration.md delete mode 100644 docs/configure_peer_server_listen_interfaces.md delete mode 100644 docs/configure_rpc_server_listen_interfaces.md create mode 100644 docs/contact.md create mode 100644 docs/controlling.md delete mode 100644 docs/default_ports.md create mode 100644 docs/developer_resources.md create mode 100644 docs/index.md create mode 100644 docs/installation.md create mode 100644 docs/make.bat create mode 100644 docs/mining.md create mode 100644 docs/requirements.txt create mode 100644 docs/table_of_content.md create mode 100644 docs/update.md delete mode 100644 docs/using_bootstrap_dat.md create mode 100644 docs/using_docker.md create mode 100644 docs/wallet.md delete mode 100644 peer/export_test.go create mode 100644 rpcclient/example_test.go create mode 100644 rpcclient/examples/bitcoincorehttpbulk/README.md create mode 100644 rpcclient/examples/bitcoincorehttpbulk/main.go create mode 100644 wire/msgsendaddrv2.go diff --git a/.github/workflows/go.yml b/.github/workflows/go.yml index f722032b16..8803194d69 100644 --- a/.github/workflows/go.yml +++ b/.github/workflows/go.yml @@ -6,7 +6,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - go: [1.13, 1.14] + go: [1.14, 1.15] steps: - name: Set up Go uses: actions/setup-go@v2 @@ -25,3 +25,8 @@ jobs: GO111MODULE: "on" run: | sh ./goclean.sh + + - name: Send coverage + uses: shogo82148/actions-goveralls@v1 + with: + path-to-profile: profile.cov diff --git a/.gitignore b/.gitignore index 72fb9416ea..c3effe5fc7 100644 --- a/.gitignore +++ b/.gitignore @@ -32,3 +32,7 @@ _cgo_export.* _testmain.go *.exe + +# Code coverage files +profile.tmp +profile.cov diff --git a/CHANGES b/CHANGES index d38897b339..fd59a88672 100644 --- a/CHANGES +++ b/CHANGES @@ -3,6 +3,126 @@ User visible changes for btcd A full-node bitcoin implementation written in Go ============================================================================ +Changes in 0.22.0 (Tue Jun 01 2021) + - Protocol and network-related changes: + - Add support for witness tx and block in notfound msg (#1625) + - Add support for receiving sendaddrv2 messages from a peer (#1670) + - Fix bug in peer package causing last block height to go backwards + (#1606) + - Add chain parameters for connecting to the public Signet network + (#1692, #1718) + - Crypto changes: + - Fix bug causing panic due to bad R and S signature components in + btcec.RecoverCompact (#1691) + - Set the name (secp256k1) in the CurveParams of the S256 curve + (#1565) + - Notable developer-related package changes: + - Remove unknown block version warning in the blockchain package, + due to false positives triggered by AsicBoost (#1463) + - Add chaincfg.RegisterHDKeyID function to populate HD key ID pairs + (#1617) + - Add new method mining.AddWitnessCommitment to add the witness + commitment as an OP_RETURN output within the coinbase transaction. + (#1716) + - RPC changes: + - Support Batch JSON-RPC in rpcclient and server (#1583) + - Add rpcclient method to invoke getdescriptorinfo JSON-RPC command + (#1578) + - Update the rpcserver handler for validateaddress JSON-RPC command to + have parity with the bitcoind 0.20.0 interface (#1613) + - Add rpcclient method to invoke getblockfilter JSON-RPC command + (#1579) + - Add signmessagewithprivkey JSON-RPC command in rpcserver (#1585) + - Add rpcclient method to invoke importmulti JSON-RPC command (#1579) + - Add watchOnly argument in rpcclient method to invoke + listtransactions JSON-RPC command (#1628) + - Update btcjson.ListTransactionsResult for compatibility with Bitcoin + Core 0.20.0 (#1626) + - Support nullable optional JSON-RPC parameters (#1594) + - Add rpcclient and server method to invoke getnodeaddresses JSON-RPC + command (#1590) + - Add rpcclient methods to invoke PSBT JSON-RPC commands (#1596) + - Add rpcclient method to invoke listsinceblock with the + include_watchonly parameter enabled (#1451) + - Add rpcclient method to invoke deriveaddresses JSON-RPC command + (#1631) + - Add rpcclient method to invoke getblocktemplate JSON-RPC command + (#1629) + - Add rpcclient method to invoke getaddressinfo JSON-RPC command + (#1633) + - Add rpcclient method to invoke getwalletinfo JSON-RPC command + (#1638) + - Fix error message in rpcserver when an unknown RPC command is + encountered (#1695) + - Fix error message returned by estimatefee when the number of blocks + exceeds the max depth (#1678) + - Update btcjson.GetBlockChainInfoResult to include new fields in + Bitcoin Core (#1676) + - Add ExtraHeaders in rpcclient.ConnConfig struct (#1669) + - Fix bitcoind compatibility issue with the sendrawtransaction + JSON-RPC command (#1659) + - Add new JSON-RPC errors to btcjson package, and documented them + (#1648) + - Add rpcclient method to invoke createwallet JSON-RPC command + (#1650) + - Add rpcclient methods to invoke backupwallet, dumpwallet, loadwallet + and unloadwallet JSON-RPC commands (#1645) + - Fix unmarshalling error in getmininginfo JSON-RPC command, for valid + integers in scientific notation (#1644) + - Add rpcclient method to invoke gettxoutsetinfo JSON-RPC command + (#1641) + - Add rpcclient method to invoke signrawtransactionwithwallet JSON-RPC + command (#1642) + - Add txid to getblocktemplate response of rpcserver (#1639) + - Fix monetary unit used in createrawtransaction JSON-RPC command in + rpcserver (#1614) + - Add rawtx field to btcjson.GetBlockVerboseTxResult to provide + backwards compatibility with older versions of Bitcoin Core (#1677) + - Misc changes: + - Update btcutil dependency (#1704) + - Add Dockerfile to build and run btcd on Docker (#1465) + - Rework documentation and publish on https://btcd.readthedocs.io (#1468) + - Add support for Go 1.15 (#1619) + - Add Go 1.14 as the minimum supported version of Golang (#1621) + - Contributors (alphabetical order): + - 10gic + - Andrew Tugarinov + - Anirudha Bose + - Appelberg-s + - Armando Ochoa + - Aurèle Oulès + - Calvin Kim + - Christian Lehmann + - Conner Fromknecht + - Dan Cline + - David Mazary + - Elliott Minns + - Federico Bond + - Friedger Müffke + - Gustavo Chain + - Hanjun Kim + - Henry Fisher + - Iskander Sharipov + - Jake Sylvestre + - Johan T. Halseth + - John C. Vernaleo + - Liran Sharir + - Mikael Lindlof + - Olaoluwa Osuntokun + - Oliver Gugger + - Rjected + - Steven Kreuzer + - Torkel Rogstad + - Tristyn + - Victor Lavaud + - Vinayak Borkar + - Wilmer Paulino + - Yaacov Akiba Slama + - ebiiim + - ipriver + - wakiyamap + - yyforyongyu + Changes in 0.21.0 (Thu Aug 27 2020) - Network-related changes: - Handle notfound messages from peers in netsync package (#1603) @@ -26,7 +146,7 @@ Changes in 0.21.0 (Thu Aug 27 2020) - Fix panic in fieldVal.SetByteSlice when called with large values, and improve the method to be 35% faster (#1602) - btcctl changes: - - Added -regtest mode to btcctl (#1556) + - Add -regtest mode to btcctl (#1556) - Misc changes: - Fix a bug due to a deadlock in connmgr's dynamic ban scoring (#1509) - Add blockchain.NewUtxoEntry() to directly create entries for diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000000..3bbc25712b --- /dev/null +++ b/Dockerfile @@ -0,0 +1,41 @@ +# This Dockerfile builds btcd from source and creates a small (55 MB) docker container based on alpine linux. +# +# Clone this repository and run the following command to build and tag a fresh btcd amd64 container: +# +# docker build . -t yourregistry/btcd +# +# You can use the following command to buid an arm64v8 container: +# +# docker build . -t yourregistry/btcd --build-arg ARCH=arm64v8 +# +# For more information how to use this docker image visit: +# https://github.com/btcsuite/btcd/tree/master/docs +# +# 8333 Mainnet Bitcoin peer-to-peer port +# 8334 Mainet RPC port + +ARG ARCH=amd64 + +FROM golang:1.14-alpine3.12 AS build-container + +ARG ARCH +ENV GO111MODULE=on + +ADD . /app +WORKDIR /app +RUN set -ex \ + && if [ "${ARCH}" = "amd64" ]; then export GOARCH=amd64; fi \ + && if [ "${ARCH}" = "arm32v7" ]; then export GOARCH=arm; fi \ + && if [ "${ARCH}" = "arm64v8" ]; then export GOARCH=arm64; fi \ + && echo "Compiling for $GOARCH" \ + && go install -v . ./cmd/... + +FROM $ARCH/alpine:3.12 + +COPY --from=build-container /go/bin /bin + +VOLUME ["/root/.btcd"] + +EXPOSE 8333 8334 + +ENTRYPOINT ["btcd"] diff --git a/README.md b/README.md index 8c9d252ad4..957369a239 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,9 @@ btcd ==== [![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) +[![Coverage Status](https://coveralls.io/repos/github/btcsuite/btcd/badge.svg?branch=master)](https://coveralls.io/github/btcsuite/btcd?branch=master) [![ISC License](https://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://godoc.org/github.com/btcsuite/btcd) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd) btcd is an alternative full node bitcoin implementation written in Go (golang). @@ -26,7 +27,7 @@ transactions based on miner requirements ("standard" transactions). One key difference between btcd and Bitcoin Core is that btcd does *NOT* include wallet functionality and this was a very intentional design decision. See the -blog entry [here](https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon) +blog entry [here](https://web.archive.org/web/20171125143919/https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon) for more details. This means you can't actually make or receive payments directly with btcd. That functionality is provided by the [btcwallet](https://github.com/btcsuite/btcwallet) and @@ -35,12 +36,10 @@ which are both under active development. ## Requirements -[Go](http://golang.org) 1.12 or newer. +[Go](http://golang.org) 1.14 or newer. ## Installation -#### Windows - MSI Available - https://github.com/btcsuite/btcd/releases #### Linux/BSD/MacOSX/POSIX - Build from Source @@ -73,10 +72,6 @@ $ GO111MODULE=on go install -v . ./cmd/... ## Updating -#### Windows - -Install a newer MSI - #### Linux/BSD/MacOSX/POSIX - Build from Source - Run the following commands to update btcd, all dependencies, and install it: @@ -93,10 +88,6 @@ btcd has several configuration options available to tweak how it runs, but all of the basic operations described in the intro section work with zero configuration. -#### Windows (Installed from MSI) - -Launch btcd from your Start menu. - #### Linux/BSD/POSIX/Source ```bash diff --git a/addrmgr/addrmanager.go b/addrmgr/addrmanager.go index a8a8fb3338..fa8f27bcae 100644 --- a/addrmgr/addrmanager.go +++ b/addrmgr/addrmanager.go @@ -30,7 +30,7 @@ import ( // AddrManager provides a concurrency safe address manager for caching potential // peers on the bitcoin network. type AddrManager struct { - mtx sync.Mutex + mtx sync.RWMutex peersFile string lookupFunc func(string) ([]net.IP, error) rand *rand.Rand @@ -645,8 +645,8 @@ func (a *AddrManager) numAddresses() int { // NumAddresses returns the number of addresses known to the address manager. func (a *AddrManager) NumAddresses() int { - a.mtx.Lock() - defer a.mtx.Unlock() + a.mtx.RLock() + defer a.mtx.RUnlock() return a.numAddresses() } @@ -654,8 +654,8 @@ func (a *AddrManager) NumAddresses() int { // NeedMoreAddresses returns whether or not the address manager needs more // addresses. func (a *AddrManager) NeedMoreAddresses() bool { - a.mtx.Lock() - defer a.mtx.Unlock() + a.mtx.RLock() + defer a.mtx.RUnlock() return a.numAddresses() < needAddressThreshold } @@ -685,8 +685,8 @@ func (a *AddrManager) AddressCache() []*wire.NetAddress { // getAddresses returns all of the addresses currently found within the // manager's address cache. func (a *AddrManager) getAddresses() []*wire.NetAddress { - a.mtx.Lock() - defer a.mtx.Unlock() + a.mtx.RLock() + defer a.mtx.RUnlock() addrIndexLen := len(a.addrIndex) if addrIndexLen == 0 { diff --git a/blockchain/README.md b/blockchain/README.md index de5611bcae..2237780c10 100644 --- a/blockchain/README.md +++ b/blockchain/README.md @@ -1,9 +1,9 @@ blockchain ========== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/blockchain) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain) Package blockchain implements bitcoin block handling and chain selection rules. The test coverage is currently only around 60%, but will be increasing over @@ -61,18 +61,18 @@ is by no means exhaustive: ## Examples -* [ProcessBlock Example](http://godoc.org/github.com/btcsuite/btcd/blockchain#example-BlockChain-ProcessBlock) +* [ProcessBlock Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-BlockChain-ProcessBlock) Demonstrates how to create a new chain instance and use ProcessBlock to attempt to add a block to the chain. This example intentionally attempts to insert a duplicate genesis block to illustrate how an invalid block is handled. -* [CompactToBig Example](http://godoc.org/github.com/btcsuite/btcd/blockchain#example-CompactToBig) +* [CompactToBig Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-CompactToBig) Demonstrates how to convert the compact "bits" in a block header which represent the target difficulty to a big integer and display it using the typical hex notation. -* [BigToCompact Example](http://godoc.org/github.com/btcsuite/btcd/blockchain#example-BigToCompact) +* [BigToCompact Example](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain#example-BigToCompact) Demonstrates how to convert a target difficulty into the compact "bits" in a block header which represent that target difficulty. diff --git a/blockchain/chain.go b/blockchain/chain.go index 71c1b2ee77..eea603ce8e 100644 --- a/blockchain/chain.go +++ b/blockchain/chain.go @@ -174,11 +174,7 @@ type BlockChain struct { // // unknownRulesWarned refers to warnings due to unknown rules being // activated. - // - // unknownVersionsWarned refers to warnings due to unknown versions - // being mined. - unknownRulesWarned bool - unknownVersionsWarned bool + unknownRulesWarned bool // The notifications field stores a slice of callbacks to be executed on // certain blockchain events. @@ -574,20 +570,13 @@ func (b *BlockChain) connectBlock(node *blockNode, block *btcutil.Block, "spent transaction out information") } - // No warnings about unknown rules or versions until the chain is - // current. + // No warnings about unknown rules until the chain is current. if b.isCurrent() { // Warn if any unknown new rules are either about to activate or // have already been activated. if err := b.warnUnknownRuleActivations(node); err != nil { return err } - - // Warn if a high enough percentage of the last blocks have - // unexpected versions. - if err := b.warnUnknownVersions(node); err != nil { - return err - } } // Write any block status changes to DB before updating best state. diff --git a/blockchain/chainio.go b/blockchain/chainio.go index c456c006f3..f40ba465e9 100644 --- a/blockchain/chainio.go +++ b/blockchain/chainio.go @@ -120,7 +120,7 @@ func dbFetchVersion(dbTx database.Tx, key []byte) uint32 { return 0 } - return byteOrder.Uint32(serialized[:]) + return byteOrder.Uint32(serialized) } // dbPutVersion uses an existing database transaction to update the provided @@ -943,7 +943,7 @@ func serializeBestChainState(state bestChainState) []byte { byteOrder.PutUint32(serializedData[offset:], workSumBytesLen) offset += 4 copy(serializedData[offset:], workSumBytes) - return serializedData[:] + return serializedData } // deserializeBestChainState deserializes the passed serialized best chain diff --git a/blockchain/fullblocktests/README.md b/blockchain/fullblocktests/README.md index 41f4b934e9..943989be35 100644 --- a/blockchain/fullblocktests/README.md +++ b/blockchain/fullblocktests/README.md @@ -1,9 +1,9 @@ fullblocktests ============== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/blockchain/fullblocktests) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain/fullblocktests) Package fullblocktests provides a set of full block tests to be used for testing the consensus validation rules. The tests are intended to be flexible enough to diff --git a/blockchain/indexers/README.md b/blockchain/indexers/README.md index 20a721f3f2..f48491520a 100644 --- a/blockchain/indexers/README.md +++ b/blockchain/indexers/README.md @@ -1,9 +1,9 @@ indexers ======== -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://godoc.org/github.com/btcsuite/btcd/blockchain/indexers?status.png)](http://godoc.org/github.com/btcsuite/btcd/blockchain/indexers) +[![GoDoc](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain/indexers?status.png)](https://pkg.go.dev/github.com/btcsuite/btcd/blockchain/indexers) Package indexers implements optional block chain indexes. diff --git a/blockchain/scriptval_test.go b/blockchain/scriptval_test.go index 56450b7a03..031f04801f 100644 --- a/blockchain/scriptval_test.go +++ b/blockchain/scriptval_test.go @@ -6,7 +6,6 @@ package blockchain import ( "fmt" - "runtime" "testing" "github.com/btcsuite/btcd/txscript" @@ -15,8 +14,6 @@ import ( // TestCheckBlockScripts ensures that validating the all of the scripts in a // known-good block doesn't return an error. func TestCheckBlockScripts(t *testing.T) { - runtime.GOMAXPROCS(runtime.NumCPU()) - testBlockNum := 277647 blockDataFile := fmt.Sprintf("%d.dat.bz2", testBlockNum) blocks, err := loadBlocks(blockDataFile) diff --git a/blockchain/thresholdstate.go b/blockchain/thresholdstate.go index 35a762552a..5da74a95af 100644 --- a/blockchain/thresholdstate.go +++ b/blockchain/thresholdstate.go @@ -310,7 +310,7 @@ func (b *BlockChain) deploymentState(prevNode *blockNode, deploymentID uint32) ( // initThresholdCaches initializes the threshold state caches for each warning // bit and defined deployment and provides warnings if the chain is current per -// the warnUnknownVersions and warnUnknownRuleActivations functions. +// the warnUnknownRuleActivations function. func (b *BlockChain) initThresholdCaches() error { // Initialize the warning and deployment caches by calculating the // threshold state for each of them. This will ensure the caches are @@ -335,15 +335,9 @@ func (b *BlockChain) initThresholdCaches() error { } } - // No warnings about unknown rules or versions until the chain is - // current. + // No warnings about unknown rules until the chain is current. if b.isCurrent() { - // Warn if a high enough percentage of the last blocks have - // unexpected versions. bestNode := b.bestChain.Tip() - if err := b.warnUnknownVersions(bestNode); err != nil { - return err - } // Warn if any unknown new rules are either about to activate or // have already been activated. diff --git a/blockchain/validate.go b/blockchain/validate.go index b971337fb5..f41d54e6b1 100644 --- a/blockchain/validate.go +++ b/blockchain/validate.go @@ -877,7 +877,6 @@ func CheckTransactionInputs(tx *btcutil.Tx, txHeight int32, utxoView *UtxoViewpo return 0, nil } - txHash := tx.Hash() var totalSatoshiIn int64 for txInIndex, txIn := range tx.MsgTx().TxIn { // Ensure the referenced input transaction is available. @@ -954,7 +953,7 @@ func CheckTransactionInputs(tx *btcutil.Tx, txHeight int32, utxoView *UtxoViewpo if totalSatoshiIn < totalSatoshiOut { str := fmt.Sprintf("total value of all transaction inputs for "+ "transaction %v is %v which is less than the amount "+ - "spent of %v", txHash, totalSatoshiIn, totalSatoshiOut) + "spent of %v", tx.Hash(), totalSatoshiIn, totalSatoshiOut) return 0, ruleError(ErrSpendTooHigh, str) } diff --git a/blockchain/versionbits.go b/blockchain/versionbits.go index ef2a080257..28fcde7b69 100644 --- a/blockchain/versionbits.go +++ b/blockchain/versionbits.go @@ -26,15 +26,6 @@ const ( // vbNumBits is the total number of bits available for use with the // version bits scheme. vbNumBits = 29 - - // unknownVerNumToCheck is the number of previous blocks to consider - // when checking for a threshold of unknown block versions for the - // purposes of warning the user. - unknownVerNumToCheck = 100 - - // unknownVerWarnNum is the threshold of previous blocks that have an - // unknown version to use for the purposes of warning the user. - unknownVerWarnNum = unknownVerNumToCheck / 2 ) // bitConditionChecker provides a thresholdConditionChecker which can be used to @@ -264,38 +255,3 @@ func (b *BlockChain) warnUnknownRuleActivations(node *blockNode) error { return nil } - -// warnUnknownVersions logs a warning if a high enough percentage of the last -// blocks have unexpected versions. -// -// This function MUST be called with the chain state lock held (for writes) -func (b *BlockChain) warnUnknownVersions(node *blockNode) error { - // Nothing to do if already warned. - if b.unknownVersionsWarned { - return nil - } - - // Warn if enough previous blocks have unexpected versions. - numUpgraded := uint32(0) - for i := uint32(0); i < unknownVerNumToCheck && node != nil; i++ { - expectedVersion, err := b.calcNextBlockVersion(node.parent) - if err != nil { - return err - } - if expectedVersion > vbLegacyBlockVersion && - (node.version & ^expectedVersion) != 0 { - - numUpgraded++ - } - - node = node.parent - } - if numUpgraded > unknownVerWarnNum { - log.Warn("Unknown block versions are being mined, so new " + - "rules might be in effect. Are you running the " + - "latest version of the software?") - b.unknownVersionsWarned = true - } - - return nil -} diff --git a/btcd.go b/btcd.go index c17d6ab48a..3ace182cd8 100644 --- a/btcd.go +++ b/btcd.go @@ -297,9 +297,6 @@ func loadBlockDB() (database.DB, error) { } func main() { - // Use all processor cores. - runtime.GOMAXPROCS(runtime.NumCPU()) - // Block and transaction processing can cause bursty allocations. This // limits the garbage collector from excessively overallocating during // bursts. This value was arrived at with the help of profiling live diff --git a/btcec/README.md b/btcec/README.md index 130bd200a0..a6dd2cf285 100644 --- a/btcec/README.md +++ b/btcec/README.md @@ -1,9 +1,9 @@ btcec ===== -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcec) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://godoc.org/github.com/btcsuite/btcd/btcec?status.png)](http://godoc.org/github.com/btcsuite/btcd/btcec) +[![GoDoc](https://pkg.go.dev/github.com/btcsuite/btcd/btcec?status.png)](https://pkg.go.dev/github.com/btcsuite/btcd/btcec) Package btcec implements elliptic curve cryptography needed for working with Bitcoin (secp256k1 only for now). It is designed so that it may be used with the @@ -25,19 +25,19 @@ $ go get -u github.com/btcsuite/btcd/btcec ## Examples -* [Sign Message](http://godoc.org/github.com/btcsuite/btcd/btcec#example-package--SignMessage) +* [Sign Message](https://pkg.go.dev/github.com/btcsuite/btcd/btcec#example-package--SignMessage) Demonstrates signing a message with a secp256k1 private key that is first parsed form raw bytes and serializing the generated signature. -* [Verify Signature](http://godoc.org/github.com/btcsuite/btcd/btcec#example-package--VerifySignature) +* [Verify Signature](https://pkg.go.dev/github.com/btcsuite/btcd/btcec#example-package--VerifySignature) Demonstrates verifying a secp256k1 signature against a public key that is first parsed from raw bytes. The signature is also parsed from raw bytes. -* [Encryption](http://godoc.org/github.com/btcsuite/btcd/btcec#example-package--EncryptMessage) +* [Encryption](https://pkg.go.dev/github.com/btcsuite/btcd/btcec#example-package--EncryptMessage) Demonstrates encrypting a message for a public key that is first parsed from raw bytes, then decrypting it using the corresponding private key. -* [Decryption](http://godoc.org/github.com/btcsuite/btcd/btcec#example-package--DecryptMessage) +* [Decryption](https://pkg.go.dev/github.com/btcsuite/btcd/btcec#example-package--DecryptMessage) Demonstrates decrypting a message using a private key that is first parsed from raw bytes. diff --git a/btcec/btcec.go b/btcec/btcec.go index de93a255a4..a2e20f4b31 100644 --- a/btcec/btcec.go +++ b/btcec/btcec.go @@ -930,6 +930,8 @@ func initS256() { secp256k1.Gx = fromHex("79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798") secp256k1.Gy = fromHex("483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8") secp256k1.BitSize = 256 + // Curve name taken from https://safecurves.cr.yp.to/. + secp256k1.Name = "secp256k1" secp256k1.q = new(big.Int).Div(new(big.Int).Add(secp256k1.P, big.NewInt(1)), big.NewInt(4)) secp256k1.H = 1 diff --git a/btcec/signature.go b/btcec/signature.go index deedd172d8..cdd7cedfb8 100644 --- a/btcec/signature.go +++ b/btcec/signature.go @@ -284,6 +284,25 @@ func hashToInt(hash []byte, c elliptic.Curve) *big.Int { // format and thus we match bitcoind's behaviour here. func recoverKeyFromSignature(curve *KoblitzCurve, sig *Signature, msg []byte, iter int, doChecks bool) (*PublicKey, error) { + // Parse and validate the R and S signature components. + // + // Fail if r and s are not in [1, N-1]. + if sig.R.Cmp(curve.Params().N) != -1 { + return nil, errors.New("signature R is >= curve order") + } + + if sig.R.Sign() == 0 { + return nil, errors.New("signature R is 0") + } + + if sig.S.Cmp(curve.Params().N) != -1 { + return nil, errors.New("signature S is >= curve order") + } + + if sig.S.Sign() == 0 { + return nil, errors.New("signature S is 0") + } + // 1.1 x = (n * i) + r Rx := new(big.Int).Mul(curve.Params().N, new(big.Int).SetInt64(int64(iter/2))) @@ -393,7 +412,7 @@ func SignCompact(curve *KoblitzCurve, key *PrivateKey, // RecoverCompact verifies the compact signature "signature" of "hash" for the // Koblitz curve in "curve". If the signature matches then the recovered public -// key will be returned as well as a boolen if the original key was compressed +// key will be returned as well as a boolean if the original key was compressed // or not, else an error will be returned. func RecoverCompact(curve *KoblitzCurve, signature, hash []byte) (*PublicKey, bool, error) { diff --git a/btcec/signature_test.go b/btcec/signature_test.go index d238741455..ba02a03f76 100644 --- a/btcec/signature_test.go +++ b/btcec/signature_test.go @@ -555,6 +555,40 @@ var recoveryTests = []struct { sig: "00000000000000000000000000000000000000000000000000000000000000002c0000000000000000000000000000000000000000000000000000000000000004", pub: "04A7640409AA2083FDAD38B2D8DE1263B2251799591D840653FB02DBBA503D7745FCB83D80E08A1E02896BE691EA6AFFB8A35939A646F1FC79052A744B1C82EDC3", }, + { + // Zero R value + // + // Test case contributed by Ethereum Swarm: GH-1651 + msg: "3060d2c77c1e192d62ad712fb400e04e6f779914a6876328ff3b213fa85d2012", + sig: "65000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000037a3", + err: fmt.Errorf("signature R is 0"), + }, + { + // Zero R value + // + // Test case contributed by Ethereum Swarm: GH-1651 + msg: "2bcebac60d8a78e520ae81c2ad586792df495ed429bd730dcd897b301932d054", + sig: "060000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007c", + err: fmt.Errorf("signature R is 0"), + }, + { + // R = N (curve order of secp256k1) + msg: "2bcebac60d8a78e520ae81c2ad586792df495ed429bd730dcd897b301932d054", + sig: "65fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd036414100000000000000000000000000000000000000000000000000000000000037a3", + err: fmt.Errorf("signature R is >= curve order"), + }, + { + // Zero S value + msg: "ce0677bb30baa8cf067c88db9811f4333d131bf8bcf12fe7065d211dce971008", + sig: "0190f27b8b488db00b00606796d2987f6a5f59ae62ea05effe84fef5b8b0e549980000000000000000000000000000000000000000000000000000000000000000", + err: fmt.Errorf("signature S is 0"), + }, + { + // S = N (curve order of secp256k1) + msg: "ce0677bb30baa8cf067c88db9811f4333d131bf8bcf12fe7065d211dce971008", + sig: "0190f27b8b488db00b00606796d2987f6a5f59ae62ea05effe84fef5b8b0e54998fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141", + err: fmt.Errorf("signature S is >= curve order"), + }, } func TestRecoverCompact(t *testing.T) { diff --git a/btcjson/README.md b/btcjson/README.md index b643579543..48f322635e 100644 --- a/btcjson/README.md +++ b/btcjson/README.md @@ -1,9 +1,9 @@ btcjson ======= -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/btcjson) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/btcjson) Package btcjson implements concrete types for marshalling to and from the bitcoin JSON-RPC API. A comprehensive suite of tests is provided to ensure @@ -30,17 +30,17 @@ $ go get -u github.com/btcsuite/btcd/btcjson ## Examples -* [Marshal Command](http://godoc.org/github.com/btcsuite/btcd/btcjson#example-MarshalCmd) +* [Marshal Command](https://pkg.go.dev/github.com/btcsuite/btcd/btcjson#example-MarshalCmd) Demonstrates how to create and marshal a command into a JSON-RPC request. -* [Unmarshal Command](http://godoc.org/github.com/btcsuite/btcd/btcjson#example-UnmarshalCmd) +* [Unmarshal Command](https://pkg.go.dev/github.com/btcsuite/btcd/btcjson#example-UnmarshalCmd) Demonstrates how to unmarshal a JSON-RPC request and then unmarshal the concrete request into a concrete command. -* [Marshal Response](http://godoc.org/github.com/btcsuite/btcd/btcjson#example-MarshalResponse) +* [Marshal Response](https://pkg.go.dev/github.com/btcsuite/btcd/btcjson#example-MarshalResponse) Demonstrates how to marshal a JSON-RPC response. -* [Unmarshal Response](http://godoc.org/github.com/btcsuite/btcd/btcjson#example-package--UnmarshalResponse) +* [Unmarshal Response](https://pkg.go.dev/github.com/btcsuite/btcd/btcjson#example-package--UnmarshalResponse) Demonstrates how to unmarshal a JSON-RPC response and then unmarshal the result field in the response to a concrete type. diff --git a/btcjson/btcdextcmds_test.go b/btcjson/btcdextcmds_test.go index 143ec5224f..aaa44144e4 100644 --- a/btcjson/btcdextcmds_test.go +++ b/btcjson/btcdextcmds_test.go @@ -211,7 +211,7 @@ func TestBtcdExtCmds(t *testing.T) { for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -235,7 +235,7 @@ func TestBtcdExtCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/btcwalletextcmds_test.go b/btcjson/btcwalletextcmds_test.go index 58de1c81d0..dea1c61465 100644 --- a/btcjson/btcwalletextcmds_test.go +++ b/btcjson/btcwalletextcmds_test.go @@ -145,7 +145,7 @@ func TestBtcWalletExtCmds(t *testing.T) { for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -169,7 +169,7 @@ func TestBtcWalletExtCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/chainsvrcmds.go b/btcjson/chainsvrcmds.go index d66478c305..aa1d4415da 100644 --- a/btcjson/chainsvrcmds.go +++ b/btcjson/chainsvrcmds.go @@ -11,6 +11,7 @@ import ( "encoding/hex" "encoding/json" "fmt" + "reflect" "github.com/btcsuite/btcd/wire" ) @@ -80,11 +81,65 @@ func NewCreateRawTransactionCmd(inputs []TransactionInput, amounts map[string]fl } } +// DecodeRawTransactionCmd defines the decoderawtransaction JSON-RPC command. +type DecodeRawTransactionCmd struct { + HexTx string +} + +// NewDecodeRawTransactionCmd returns a new instance which can be used to issue +// a decoderawtransaction JSON-RPC command. +func NewDecodeRawTransactionCmd(hexTx string) *DecodeRawTransactionCmd { + return &DecodeRawTransactionCmd{ + HexTx: hexTx, + } +} + +// DecodeScriptCmd defines the decodescript JSON-RPC command. +type DecodeScriptCmd struct { + HexScript string +} + +// NewDecodeScriptCmd returns a new instance which can be used to issue a +// decodescript JSON-RPC command. +func NewDecodeScriptCmd(hexScript string) *DecodeScriptCmd { + return &DecodeScriptCmd{ + HexScript: hexScript, + } +} + +// DeriveAddressesCmd defines the deriveaddresses JSON-RPC command. +type DeriveAddressesCmd struct { + Descriptor string + Range *DescriptorRange +} + +// NewDeriveAddressesCmd returns a new instance which can be used to issue a +// deriveaddresses JSON-RPC command. +func NewDeriveAddressesCmd(descriptor string, descriptorRange *DescriptorRange) *DeriveAddressesCmd { + return &DeriveAddressesCmd{ + Descriptor: descriptor, + Range: descriptorRange, + } +} + +// ChangeType defines the different output types to use for the change address +// of a transaction built by the node. +type ChangeType string + +var ( + // ChangeTypeLegacy indicates a P2PKH change address type. + ChangeTypeLegacy ChangeType = "legacy" + // ChangeTypeP2SHSegWit indicates a P2WPKH-in-P2SH change address type. + ChangeTypeP2SHSegWit ChangeType = "p2sh-segwit" + // ChangeTypeBech32 indicates a P2WPKH change address type. + ChangeTypeBech32 ChangeType = "bech32" +) + // FundRawTransactionOpts are the different options that can be passed to rawtransaction type FundRawTransactionOpts struct { ChangeAddress *string `json:"changeAddress,omitempty"` ChangePosition *int `json:"changePosition,omitempty"` - ChangeType *string `json:"change_type,omitempty"` + ChangeType *ChangeType `json:"change_type,omitempty"` IncludeWatching *bool `json:"includeWatching,omitempty"` LockUnspents *bool `json:"lockUnspents,omitempty"` FeeRate *float64 `json:"feeRate,omitempty"` // BTC/kB @@ -111,32 +166,6 @@ func NewFundRawTransactionCmd(serializedTx []byte, opts FundRawTransactionOpts, } } -// DecodeRawTransactionCmd defines the decoderawtransaction JSON-RPC command. -type DecodeRawTransactionCmd struct { - HexTx string -} - -// NewDecodeRawTransactionCmd returns a new instance which can be used to issue -// a decoderawtransaction JSON-RPC command. -func NewDecodeRawTransactionCmd(hexTx string) *DecodeRawTransactionCmd { - return &DecodeRawTransactionCmd{ - HexTx: hexTx, - } -} - -// DecodeScriptCmd defines the decodescript JSON-RPC command. -type DecodeScriptCmd struct { - HexScript string -} - -// NewDecodeScriptCmd returns a new instance which can be used to issue a -// decodescript JSON-RPC command. -func NewDecodeScriptCmd(hexScript string) *DecodeScriptCmd { - return &DecodeScriptCmd{ - HexScript: hexScript, - } -} - // GetAddedNodeInfoCmd defines the getaddednodeinfo JSON-RPC command. type GetAddedNodeInfoCmd struct { DNS bool @@ -200,6 +229,33 @@ func NewGetBlockCountCmd() *GetBlockCountCmd { return &GetBlockCountCmd{} } +// FilterTypeName defines the type used in the getblockfilter JSON-RPC command for the +// filter type field. +type FilterTypeName string + +const ( + // FilterTypeBasic is the basic filter type defined in BIP0158. + FilterTypeBasic FilterTypeName = "basic" +) + +// GetBlockFilterCmd defines the getblockfilter JSON-RPC command. +type GetBlockFilterCmd struct { + BlockHash string // The hash of the block + FilterType *FilterTypeName // The type name of the filter, default=basic +} + +// NewGetBlockFilterCmd returns a new instance which can be used to issue a +// getblockfilter JSON-RPC command. +// +// The parameters which are pointers indicate they are optional. Passing nil +// for optional parameters will use the default value. +func NewGetBlockFilterCmd(blockHash string, filterType *FilterTypeName) *GetBlockFilterCmd { + return &GetBlockFilterCmd{ + BlockHash: blockHash, + FilterType: filterType, + } +} + // GetBlockHashCmd defines the getblockhash JSON-RPC command. type GetBlockHashCmd struct { Index int64 @@ -295,6 +351,10 @@ type TemplateRequest struct { // "proposal". Data string `json:"data,omitempty"` WorkID string `json:"workid,omitempty"` + + // list of supported softfork deployments, by name + // Ref: https://en.bitcoin.it/wiki/BIP_0009#getblocktemplate_changes. + Rules []string `json:"rules,omitempty"` } // convertTemplateRequestField potentially converts the provided value as @@ -427,6 +487,19 @@ func NewGetConnectionCountCmd() *GetConnectionCountCmd { return &GetConnectionCountCmd{} } +// GetDescriptorInfoCmd defines the getdescriptorinfo JSON-RPC command. +type GetDescriptorInfoCmd struct { + Descriptor string +} + +// NewGetDescriptorInfoCmd returns a new instance which can be used to issue a +// getdescriptorinfo JSON-RPC command. +func NewGetDescriptorInfoCmd(descriptor string) *GetDescriptorInfoCmd { + return &GetDescriptorInfoCmd{ + Descriptor: descriptor, + } +} + // GetDifficultyCmd defines the getdifficulty JSON-RPC command. type GetDifficultyCmd struct{} @@ -530,6 +603,22 @@ func NewGetNetworkHashPSCmd(numBlocks, height *int) *GetNetworkHashPSCmd { } } +// GetNodeAddressesCmd defines the getnodeaddresses JSON-RPC command. +type GetNodeAddressesCmd struct { + Count *int32 `jsonrpcdefault:"1"` +} + +// NewGetNodeAddressesCmd returns a new instance which can be used to issue a +// getnodeaddresses JSON-RPC command. +// +// The parameters which are pointers indicate they are optional. Passing nil +// for optional parameters will use the default value. +func NewGetNodeAddressesCmd(count *int32) *GetNodeAddressesCmd { + return &GetNodeAddressesCmd{ + Count: count, + } +} + // GetPeerInfoCmd defines the getpeerinfo JSON-RPC command. type GetPeerInfoCmd struct{} @@ -731,11 +820,60 @@ func NewSearchRawTransactionsCmd(address string, verbose, skip, count *int, vinE } } +// AllowHighFeesOrMaxFeeRate defines a type that can either be the legacy +// allowhighfees boolean field or the new maxfeerate int field. +type AllowHighFeesOrMaxFeeRate struct { + Value interface{} +} + +// String returns the string representation of this struct, used for printing +// the marshaled default value in the help text. +func (a AllowHighFeesOrMaxFeeRate) String() string { + b, _ := a.MarshalJSON() + return string(b) +} + +// MarshalJSON implements the json.Marshaler interface +func (a AllowHighFeesOrMaxFeeRate) MarshalJSON() ([]byte, error) { + // The default value is false which only works with the legacy versions. + if a.Value == nil || + (reflect.ValueOf(a.Value).Kind() == reflect.Ptr && + reflect.ValueOf(a.Value).IsNil()) { + + return json.Marshal(false) + } + + return json.Marshal(a.Value) +} + +// UnmarshalJSON implements the json.Unmarshaler interface +func (a *AllowHighFeesOrMaxFeeRate) UnmarshalJSON(data []byte) error { + if len(data) == 0 { + return nil + } + + var unmarshalled interface{} + if err := json.Unmarshal(data, &unmarshalled); err != nil { + return err + } + + switch v := unmarshalled.(type) { + case bool: + a.Value = Bool(v) + case float64: + a.Value = Int32(int32(v)) + default: + return fmt.Errorf("invalid allowhighfees or maxfeerate value: "+ + "%v", unmarshalled) + } + + return nil +} + // SendRawTransactionCmd defines the sendrawtransaction JSON-RPC command. type SendRawTransactionCmd struct { - HexTx string - AllowHighFees *bool `jsonrpcdefault:"false"` - MaxFeeRate *int32 + HexTx string + FeeSetting *AllowHighFeesOrMaxFeeRate `jsonrpcdefault:"false"` } // NewSendRawTransactionCmd returns a new instance which can be used to issue a @@ -745,8 +883,10 @@ type SendRawTransactionCmd struct { // for optional parameters will use the default value. func NewSendRawTransactionCmd(hexTx string, allowHighFees *bool) *SendRawTransactionCmd { return &SendRawTransactionCmd{ - HexTx: hexTx, - AllowHighFees: allowHighFees, + HexTx: hexTx, + FeeSetting: &AllowHighFeesOrMaxFeeRate{ + Value: allowHighFees, + }, } } @@ -756,8 +896,10 @@ func NewSendRawTransactionCmd(hexTx string, allowHighFees *bool) *SendRawTransac // A 0 maxFeeRate indicates that a maximum fee rate won't be enforced. func NewBitcoindSendRawTransactionCmd(hexTx string, maxFeeRate int32) *SendRawTransactionCmd { return &SendRawTransactionCmd{ - HexTx: hexTx, - MaxFeeRate: &maxFeeRate, + HexTx: hexTx, + FeeSetting: &AllowHighFeesOrMaxFeeRate{ + Value: &maxFeeRate, + }, } } @@ -779,6 +921,24 @@ func NewSetGenerateCmd(generate bool, genProcLimit *int) *SetGenerateCmd { } } +// SignMessageWithPrivKeyCmd defines the signmessagewithprivkey JSON-RPC command. +type SignMessageWithPrivKeyCmd struct { + PrivKey string // base 58 Wallet Import format private key + Message string // Message to sign +} + +// NewSignMessageWithPrivKey returns a new instance which can be used to issue a +// signmessagewithprivkey JSON-RPC command. +// +// The first parameter is a private key in base 58 Wallet Import format. +// The second parameter is the message to sign. +func NewSignMessageWithPrivKey(privKey, message string) *SignMessageWithPrivKeyCmd { + return &SignMessageWithPrivKeyCmd{ + PrivKey: privKey, + Message: message, + } +} + // StopCmd defines the stop JSON-RPC command. type StopCmd struct{} @@ -888,14 +1048,16 @@ func init() { MustRegisterCmd("addnode", (*AddNodeCmd)(nil), flags) MustRegisterCmd("createrawtransaction", (*CreateRawTransactionCmd)(nil), flags) - MustRegisterCmd("fundrawtransaction", (*FundRawTransactionCmd)(nil), flags) MustRegisterCmd("decoderawtransaction", (*DecodeRawTransactionCmd)(nil), flags) MustRegisterCmd("decodescript", (*DecodeScriptCmd)(nil), flags) + MustRegisterCmd("deriveaddresses", (*DeriveAddressesCmd)(nil), flags) + MustRegisterCmd("fundrawtransaction", (*FundRawTransactionCmd)(nil), flags) MustRegisterCmd("getaddednodeinfo", (*GetAddedNodeInfoCmd)(nil), flags) MustRegisterCmd("getbestblockhash", (*GetBestBlockHashCmd)(nil), flags) MustRegisterCmd("getblock", (*GetBlockCmd)(nil), flags) MustRegisterCmd("getblockchaininfo", (*GetBlockChainInfoCmd)(nil), flags) MustRegisterCmd("getblockcount", (*GetBlockCountCmd)(nil), flags) + MustRegisterCmd("getblockfilter", (*GetBlockFilterCmd)(nil), flags) MustRegisterCmd("getblockhash", (*GetBlockHashCmd)(nil), flags) MustRegisterCmd("getblockheader", (*GetBlockHeaderCmd)(nil), flags) MustRegisterCmd("getblockstats", (*GetBlockStatsCmd)(nil), flags) @@ -905,6 +1067,7 @@ func init() { MustRegisterCmd("getchaintips", (*GetChainTipsCmd)(nil), flags) MustRegisterCmd("getchaintxstats", (*GetChainTxStatsCmd)(nil), flags) MustRegisterCmd("getconnectioncount", (*GetConnectionCountCmd)(nil), flags) + MustRegisterCmd("getdescriptorinfo", (*GetDescriptorInfoCmd)(nil), flags) MustRegisterCmd("getdifficulty", (*GetDifficultyCmd)(nil), flags) MustRegisterCmd("getgenerate", (*GetGenerateCmd)(nil), flags) MustRegisterCmd("gethashespersec", (*GetHashesPerSecCmd)(nil), flags) @@ -915,6 +1078,7 @@ func init() { MustRegisterCmd("getnetworkinfo", (*GetNetworkInfoCmd)(nil), flags) MustRegisterCmd("getnettotals", (*GetNetTotalsCmd)(nil), flags) MustRegisterCmd("getnetworkhashps", (*GetNetworkHashPSCmd)(nil), flags) + MustRegisterCmd("getnodeaddresses", (*GetNodeAddressesCmd)(nil), flags) MustRegisterCmd("getpeerinfo", (*GetPeerInfoCmd)(nil), flags) MustRegisterCmd("getrawmempool", (*GetRawMempoolCmd)(nil), flags) MustRegisterCmd("getrawtransaction", (*GetRawTransactionCmd)(nil), flags) @@ -930,6 +1094,7 @@ func init() { MustRegisterCmd("searchrawtransactions", (*SearchRawTransactionsCmd)(nil), flags) MustRegisterCmd("sendrawtransaction", (*SendRawTransactionCmd)(nil), flags) MustRegisterCmd("setgenerate", (*SetGenerateCmd)(nil), flags) + MustRegisterCmd("signmessagewithprivkey", (*SignMessageWithPrivKeyCmd)(nil), flags) MustRegisterCmd("stop", (*StopCmd)(nil), flags) MustRegisterCmd("submitblock", (*SubmitBlockCmd)(nil), flags) MustRegisterCmd("uptime", (*UptimeCmd)(nil), flags) diff --git a/btcjson/chainsvrcmds_test.go b/btcjson/chainsvrcmds_test.go index e2b5025727..7d3a68dc41 100644 --- a/btcjson/chainsvrcmds_test.go +++ b/btcjson/chainsvrcmds_test.go @@ -127,7 +127,7 @@ func TestChainSvrCmds(t *testing.T) { } changeAddress := "bcrt1qeeuctq9wutlcl5zatge7rjgx0k45228cxez655" change := 1 - changeType := "legacy" + changeType := btcjson.ChangeTypeLegacy watching := true lockUnspents := true feeRate := 0.7 @@ -151,7 +151,7 @@ func TestChainSvrCmds(t *testing.T) { unmarshalled: func() interface{} { changeAddress := "bcrt1qeeuctq9wutlcl5zatge7rjgx0k45228cxez655" change := 1 - changeType := "legacy" + changeType := btcjson.ChangeTypeLegacy watching := true lockUnspents := true feeRate := 0.7 @@ -220,6 +220,51 @@ func TestChainSvrCmds(t *testing.T) { marshalled: `{"jsonrpc":"1.0","method":"decodescript","params":["00"],"id":1}`, unmarshalled: &btcjson.DecodeScriptCmd{HexScript: "00"}, }, + { + name: "deriveaddresses no range", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("deriveaddresses", "00") + }, + staticCmd: func() interface{} { + return btcjson.NewDeriveAddressesCmd("00", nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"deriveaddresses","params":["00"],"id":1}`, + unmarshalled: &btcjson.DeriveAddressesCmd{Descriptor: "00"}, + }, + { + name: "deriveaddresses int range", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "deriveaddresses", "00", btcjson.DescriptorRange{Value: 2}) + }, + staticCmd: func() interface{} { + return btcjson.NewDeriveAddressesCmd( + "00", &btcjson.DescriptorRange{Value: 2}) + }, + marshalled: `{"jsonrpc":"1.0","method":"deriveaddresses","params":["00",2],"id":1}`, + unmarshalled: &btcjson.DeriveAddressesCmd{ + Descriptor: "00", + Range: &btcjson.DescriptorRange{Value: 2}, + }, + }, + { + name: "deriveaddresses slice range", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "deriveaddresses", "00", + btcjson.DescriptorRange{Value: []int{0, 2}}, + ) + }, + staticCmd: func() interface{} { + return btcjson.NewDeriveAddressesCmd( + "00", &btcjson.DescriptorRange{Value: []int{0, 2}}) + }, + marshalled: `{"jsonrpc":"1.0","method":"deriveaddresses","params":["00",[0,2]],"id":1}`, + unmarshalled: &btcjson.DeriveAddressesCmd{ + Descriptor: "00", + Range: &btcjson.DescriptorRange{Value: []int{0, 2}}, + }, + }, { name: "getaddednodeinfo", newCmd: func() (interface{}, error) { @@ -334,6 +379,28 @@ func TestChainSvrCmds(t *testing.T) { marshalled: `{"jsonrpc":"1.0","method":"getblockcount","params":[],"id":1}`, unmarshalled: &btcjson.GetBlockCountCmd{}, }, + { + name: "getblockfilter", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getblockfilter", "0000afaf") + }, + staticCmd: func() interface{} { + return btcjson.NewGetBlockFilterCmd("0000afaf", nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"getblockfilter","params":["0000afaf"],"id":1}`, + unmarshalled: &btcjson.GetBlockFilterCmd{"0000afaf", nil}, + }, + { + name: "getblockfilter optional filtertype", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getblockfilter", "0000afaf", "basic") + }, + staticCmd: func() interface{} { + return btcjson.NewGetBlockFilterCmd("0000afaf", btcjson.NewFilterTypeName(btcjson.FilterTypeBasic)) + }, + marshalled: `{"jsonrpc":"1.0","method":"getblockfilter","params":["0000afaf","basic"],"id":1}`, + unmarshalled: &btcjson.GetBlockFilterCmd{"0000afaf", btcjson.NewFilterTypeName(btcjson.FilterTypeBasic)}, + }, { name: "getblockhash", newCmd: func() (interface{}, error) { @@ -731,6 +798,32 @@ func TestChainSvrCmds(t *testing.T) { Height: btcjson.Int(123), }, }, + { + name: "getnodeaddresses", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getnodeaddresses") + }, + staticCmd: func() interface{} { + return btcjson.NewGetNodeAddressesCmd(nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"getnodeaddresses","params":[],"id":1}`, + unmarshalled: &btcjson.GetNodeAddressesCmd{ + Count: btcjson.Int32(1), + }, + }, + { + name: "getnodeaddresses optional", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getnodeaddresses", 10) + }, + staticCmd: func() interface{} { + return btcjson.NewGetNodeAddressesCmd(btcjson.Int32(10)) + }, + marshalled: `{"jsonrpc":"1.0","method":"getnodeaddresses","params":[10],"id":1}`, + unmarshalled: &btcjson.GetNodeAddressesCmd{ + Count: btcjson.Int32(10), + }, + }, { name: "getpeerinfo", newCmd: func() (interface{}, error) { @@ -1108,32 +1201,72 @@ func TestChainSvrCmds(t *testing.T) { FilterAddrs: &[]string{"1Address"}, }, }, + { + name: "searchrawtransactions", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("searchrawtransactions", "1Address", 0, 5, 10, "null", true, []string{"1Address"}) + }, + staticCmd: func() interface{} { + return btcjson.NewSearchRawTransactionsCmd("1Address", + btcjson.Int(0), btcjson.Int(5), btcjson.Int(10), nil, btcjson.Bool(true), &[]string{"1Address"}) + }, + marshalled: `{"jsonrpc":"1.0","method":"searchrawtransactions","params":["1Address",0,5,10,null,true,["1Address"]],"id":1}`, + unmarshalled: &btcjson.SearchRawTransactionsCmd{ + Address: "1Address", + Verbose: btcjson.Int(0), + Skip: btcjson.Int(5), + Count: btcjson.Int(10), + VinExtra: nil, + Reverse: btcjson.Bool(true), + FilterAddrs: &[]string{"1Address"}, + }, + }, { name: "sendrawtransaction", newCmd: func() (interface{}, error) { - return btcjson.NewCmd("sendrawtransaction", "1122") + return btcjson.NewCmd("sendrawtransaction", "1122", &btcjson.AllowHighFeesOrMaxFeeRate{}) }, staticCmd: func() interface{} { return btcjson.NewSendRawTransactionCmd("1122", nil) }, - marshalled: `{"jsonrpc":"1.0","method":"sendrawtransaction","params":["1122"],"id":1}`, + marshalled: `{"jsonrpc":"1.0","method":"sendrawtransaction","params":["1122",false],"id":1}`, unmarshalled: &btcjson.SendRawTransactionCmd{ - HexTx: "1122", - AllowHighFees: btcjson.Bool(false), + HexTx: "1122", + FeeSetting: &btcjson.AllowHighFeesOrMaxFeeRate{ + Value: btcjson.Bool(false), + }, }, }, { name: "sendrawtransaction optional", newCmd: func() (interface{}, error) { - return btcjson.NewCmd("sendrawtransaction", "1122", false) + return btcjson.NewCmd("sendrawtransaction", "1122", &btcjson.AllowHighFeesOrMaxFeeRate{Value: btcjson.Bool(false)}) }, staticCmd: func() interface{} { return btcjson.NewSendRawTransactionCmd("1122", btcjson.Bool(false)) }, marshalled: `{"jsonrpc":"1.0","method":"sendrawtransaction","params":["1122",false],"id":1}`, unmarshalled: &btcjson.SendRawTransactionCmd{ - HexTx: "1122", - AllowHighFees: btcjson.Bool(false), + HexTx: "1122", + FeeSetting: &btcjson.AllowHighFeesOrMaxFeeRate{ + Value: btcjson.Bool(false), + }, + }, + }, + { + name: "sendrawtransaction optional, bitcoind >= 0.19.0", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("sendrawtransaction", "1122", &btcjson.AllowHighFeesOrMaxFeeRate{Value: btcjson.Int32(1234)}) + }, + staticCmd: func() interface{} { + return btcjson.NewBitcoindSendRawTransactionCmd("1122", 1234) + }, + marshalled: `{"jsonrpc":"1.0","method":"sendrawtransaction","params":["1122",1234],"id":1}`, + unmarshalled: &btcjson.SendRawTransactionCmd{ + HexTx: "1122", + FeeSetting: &btcjson.AllowHighFeesOrMaxFeeRate{ + Value: btcjson.Int32(1234), + }, }, }, { @@ -1164,6 +1297,20 @@ func TestChainSvrCmds(t *testing.T) { GenProcLimit: btcjson.Int(6), }, }, + { + name: "signmessagewithprivkey", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("signmessagewithprivkey", "5Hue", "Hey") + }, + staticCmd: func() interface{} { + return btcjson.NewSignMessageWithPrivKey("5Hue", "Hey") + }, + marshalled: `{"jsonrpc":"1.0","method":"signmessagewithprivkey","params":["5Hue","Hey"],"id":1}`, + unmarshalled: &btcjson.SignMessageWithPrivKeyCmd{ + PrivKey: "5Hue", + Message: "Hey", + }, + }, { name: "stop", newCmd: func() (interface{}, error) { @@ -1302,13 +1449,24 @@ func TestChainSvrCmds(t *testing.T) { Proof: "test", }, }, + { + name: "getdescriptorinfo", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getdescriptorinfo", "123") + }, + staticCmd: func() interface{} { + return btcjson.NewGetDescriptorInfoCmd("123") + }, + marshalled: `{"jsonrpc":"1.0","method":"getdescriptorinfo","params":["123"],"id":1}`, + unmarshalled: &btcjson.GetDescriptorInfoCmd{Descriptor: "123"}, + }, } t.Logf("Running %d tests", len(tests)) for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -1333,7 +1491,7 @@ func TestChainSvrCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/chainsvrresults.go b/btcjson/chainsvrresults.go index 0bbd8679d7..aae8c645c2 100644 --- a/btcjson/chainsvrresults.go +++ b/btcjson/chainsvrresults.go @@ -9,6 +9,8 @@ import ( "encoding/hex" "encoding/json" + "github.com/btcsuite/btcd/chaincfg/chainhash" + "github.com/btcsuite/btcd/wire" "github.com/btcsuite/btcutil" ) @@ -106,6 +108,7 @@ type GetBlockVerboseTxResult struct { VersionHex string `json:"versionHex"` MerkleRoot string `json:"merkleroot"` Tx []TxRawResult `json:"tx,omitempty"` + RawTx []TxRawResult `json:"rawtx,omitempty"` // Deprecated: removed in Bitcoin Core Time int64 `json:"time"` Nonce uint32 `json:"nonce"` Bits string `json:"bits"` @@ -219,18 +222,28 @@ type GetBlockChainInfoResult struct { Difficulty float64 `json:"difficulty"` MedianTime int64 `json:"mediantime"` VerificationProgress float64 `json:"verificationprogress,omitempty"` + InitialBlockDownload bool `json:"initialblockdownload,omitempty"` Pruned bool `json:"pruned"` PruneHeight int32 `json:"pruneheight,omitempty"` ChainWork string `json:"chainwork,omitempty"` + SizeOnDisk int64 `json:"size_on_disk,omitempty"` *SoftForks *UnifiedSoftForks } +// GetBlockFilterResult models the data returned from the getblockfilter +// command. +type GetBlockFilterResult struct { + Filter string `json:"filter"` // the hex-encoded filter data + Header string `json:"header"` // the hex-encoded filter header +} + // GetBlockTemplateResultTx models the transactions field of the // getblocktemplate command. type GetBlockTemplateResultTx struct { Data string `json:"data"` Hash string `json:"hash"` + TxID string `json:"txid"` Depends []int64 `json:"depends"` Fee int64 `json:"fee"` SigOps int64 `json:"sigops"` @@ -358,6 +371,16 @@ type GetNetworkInfoResult struct { Warnings string `json:"warnings"` } +// GetNodeAddressesResult models the data returned from the getnodeaddresses +// command. +type GetNodeAddressesResult struct { + // Timestamp in seconds since epoch (Jan 1 1970 GMT) keeping track of when the node was last seen + Time int64 `json:"time"` + Services uint64 `json:"services"` // The services offered + Address string `json:"address"` // The address of the node + Port uint16 `json:"port"` // The port of the node +} + // GetPeerInfoResult models the data returned from the getpeerinfo command. type GetPeerInfoResult struct { ID int32 `json:"id"` @@ -417,6 +440,64 @@ type GetTxOutResult struct { Coinbase bool `json:"coinbase"` } +// GetTxOutSetInfoResult models the data from the gettxoutsetinfo command. +type GetTxOutSetInfoResult struct { + Height int64 `json:"height"` + BestBlock chainhash.Hash `json:"bestblock"` + Transactions int64 `json:"transactions"` + TxOuts int64 `json:"txouts"` + BogoSize int64 `json:"bogosize"` + HashSerialized chainhash.Hash `json:"hash_serialized_2"` + DiskSize int64 `json:"disk_size"` + TotalAmount btcutil.Amount `json:"total_amount"` +} + +// UnmarshalJSON unmarshals the result of the gettxoutsetinfo JSON-RPC call +func (g *GetTxOutSetInfoResult) UnmarshalJSON(data []byte) error { + // Step 1: Create type aliases of the original struct. + type Alias GetTxOutSetInfoResult + + // Step 2: Create an anonymous struct with raw replacements for the special + // fields. + aux := &struct { + BestBlock string `json:"bestblock"` + HashSerialized string `json:"hash_serialized_2"` + TotalAmount float64 `json:"total_amount"` + *Alias + }{ + Alias: (*Alias)(g), + } + + // Step 3: Unmarshal the data into the anonymous struct. + if err := json.Unmarshal(data, &aux); err != nil { + return err + } + + // Step 4: Convert the raw fields to the desired types + blockHash, err := chainhash.NewHashFromStr(aux.BestBlock) + if err != nil { + return err + } + + g.BestBlock = *blockHash + + serializedHash, err := chainhash.NewHashFromStr(aux.HashSerialized) + if err != nil { + return err + } + + g.HashSerialized = *serializedHash + + amount, err := btcutil.NewAmount(aux.TotalAmount) + if err != nil { + return err + } + + g.TotalAmount = amount + + return nil +} + // GetNetTotalsResult models the data returned from the getnettotals command. type GetNetTotalsResult struct { TotalBytesRecv uint64 `json:"totalbytesrecv"` @@ -595,8 +676,8 @@ type GetMiningInfoResult struct { Errors string `json:"errors"` Generate bool `json:"generate"` GenProcLimit int32 `json:"genproclimit"` - HashesPerSec int64 `json:"hashespersec"` - NetworkHashPS int64 `json:"networkhashps"` + HashesPerSec float64 `json:"hashespersec"` + NetworkHashPS float64 `json:"networkhashps"` PooledTx uint64 `json:"pooledtx"` TestNet bool `json:"testnet"` } @@ -671,9 +752,17 @@ type TxRawDecodeResult struct { // ValidateAddressChainResult models the data returned by the chain server // validateaddress command. +// +// Compared to the Bitcoin Core version, this struct lacks the scriptPubKey +// field since it requires wallet access, which is outside the scope of btcd. +// Ref: https://bitcoincore.org/en/doc/0.20.0/rpc/util/validateaddress/ type ValidateAddressChainResult struct { - IsValid bool `json:"isvalid"` - Address string `json:"address,omitempty"` + IsValid bool `json:"isvalid"` + Address string `json:"address,omitempty"` + IsScript *bool `json:"isscript,omitempty"` + IsWitness *bool `json:"iswitness,omitempty"` + WitnessVersion *int32 `json:"witness_version,omitempty"` + WitnessProgram *string `json:"witness_program,omitempty"` } // EstimateSmartFeeResult models the data returned buy the chain server @@ -730,3 +819,26 @@ func (f *FundRawTransactionResult) UnmarshalJSON(data []byte) error { f.ChangePosition = rawRes.ChangePosition return nil } + +// GetDescriptorInfoResult models the data from the getdescriptorinfo command. +type GetDescriptorInfoResult struct { + Descriptor string `json:"descriptor"` // descriptor in canonical form, without private keys + Checksum string `json:"checksum"` // checksum for the input descriptor + IsRange bool `json:"isrange"` // whether the descriptor is ranged + IsSolvable bool `json:"issolvable"` // whether the descriptor is solvable + HasPrivateKeys bool `json:"hasprivatekeys"` // whether the descriptor has at least one private key +} + +// DeriveAddressesResult models the data from the deriveaddresses command. +type DeriveAddressesResult []string + +// LoadWalletResult models the data from the loadwallet command +type LoadWalletResult struct { + Name string `json:"name"` + Warning string `json:"warning"` +} + +// DumpWalletResult models the data from the dumpwallet command +type DumpWalletResult struct { + Filename string `json:"filename"` +} diff --git a/btcjson/chainsvrresults_test.go b/btcjson/chainsvrresults_test.go index 1d568e2658..72dcd8d7ec 100644 --- a/btcjson/chainsvrresults_test.go +++ b/btcjson/chainsvrresults_test.go @@ -6,9 +6,13 @@ package btcjson_test import ( "encoding/json" + "reflect" "testing" "github.com/btcsuite/btcd/btcjson" + "github.com/btcsuite/btcd/chaincfg/chainhash" + "github.com/btcsuite/btcutil" + "github.com/davecgh/go-spew/spew" ) // TestChainSvrCustomResults ensures any results that have custom marshalling @@ -86,3 +90,112 @@ func TestChainSvrCustomResults(t *testing.T) { } } } + +// TestGetTxOutSetInfoResult ensures that custom unmarshalling of +// GetTxOutSetInfoResult works as intended. +func TestGetTxOutSetInfoResult(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + result string + want btcjson.GetTxOutSetInfoResult + }{ + { + name: "GetTxOutSetInfoResult - not scanning", + result: `{"height":123,"bestblock":"000000000000005f94116250e2407310463c0a7cf950f1af9ebe935b1c0687ab","transactions":1,"txouts":1,"bogosize":1,"hash_serialized_2":"9a0a561203ff052182993bc5d0cb2c620880bfafdbd80331f65fd9546c3e5c3e","disk_size":1,"total_amount":0.2}`, + want: btcjson.GetTxOutSetInfoResult{ + Height: 123, + BestBlock: func() chainhash.Hash { + h, err := chainhash.NewHashFromStr("000000000000005f94116250e2407310463c0a7cf950f1af9ebe935b1c0687ab") + if err != nil { + panic(err) + } + + return *h + }(), + Transactions: 1, + TxOuts: 1, + BogoSize: 1, + HashSerialized: func() chainhash.Hash { + h, err := chainhash.NewHashFromStr("9a0a561203ff052182993bc5d0cb2c620880bfafdbd80331f65fd9546c3e5c3e") + if err != nil { + panic(err) + } + + return *h + }(), + DiskSize: 1, + TotalAmount: func() btcutil.Amount { + a, err := btcutil.NewAmount(0.2) + if err != nil { + panic(err) + } + + return a + }(), + }, + }, + } + + t.Logf("Running %d tests", len(tests)) + for i, test := range tests { + var out btcjson.GetTxOutSetInfoResult + err := json.Unmarshal([]byte(test.result), &out) + if err != nil { + t.Errorf("Test #%d (%s) unexpected error: %v", i, + test.name, err) + continue + } + + if !reflect.DeepEqual(out, test.want) { + t.Errorf("Test #%d (%s) unexpected unmarshalled data - "+ + "got %v, want %v", i, test.name, spew.Sdump(out), + spew.Sdump(test.want)) + continue + } + } +} + +// TestChainSvrMiningInfoResults ensures GetMiningInfoResults are unmarshalled correctly +func TestChainSvrMiningInfoResults(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + result string + expected btcjson.GetMiningInfoResult + }{ + { + name: "mining info with integer networkhashps", + result: `{"networkhashps": 89790618491361}`, + expected: btcjson.GetMiningInfoResult{ + NetworkHashPS: 89790618491361, + }, + }, + { + name: "mining info with scientific notation networkhashps", + result: `{"networkhashps": 8.9790618491361e+13}`, + expected: btcjson.GetMiningInfoResult{ + NetworkHashPS: 89790618491361, + }, + }, + } + + t.Logf("Running %d tests", len(tests)) + for i, test := range tests { + var miningInfoResult btcjson.GetMiningInfoResult + err := json.Unmarshal([]byte(test.result), &miningInfoResult) + if err != nil { + t.Errorf("Test #%d (%s) unexpected error: %v", i, + test.name, err) + continue + } + if miningInfoResult != test.expected { + t.Errorf("Test #%d (%s) unexpected marhsalled data - "+ + "got %+v, want %+v", i, test.name, miningInfoResult, + test.expected) + continue + } + } +} diff --git a/btcjson/chainsvrwscmds_test.go b/btcjson/chainsvrwscmds_test.go index b0cd63cc61..03fb22c8e0 100644 --- a/btcjson/chainsvrwscmds_test.go +++ b/btcjson/chainsvrwscmds_test.go @@ -233,7 +233,7 @@ func TestChainSvrWsCmds(t *testing.T) { for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -257,7 +257,7 @@ func TestChainSvrWsCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/chainsvrwsntfns_test.go b/btcjson/chainsvrwsntfns_test.go index 2da1e7ad2f..e2b234c2a0 100644 --- a/btcjson/chainsvrwsntfns_test.go +++ b/btcjson/chainsvrwsntfns_test.go @@ -231,7 +231,7 @@ func TestChainSvrWsNtfns(t *testing.T) { for i, test := range tests { // Marshal the notification as created by the new static // creation function. The ID is nil for notifications. - marshalled, err := btcjson.MarshalCmd(nil, test.staticNtfn()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, test.staticNtfn()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -256,7 +256,7 @@ func TestChainSvrWsNtfns(t *testing.T) { // Marshal the notification as created by the generic new // notification creation function. The ID is nil for // notifications. - marshalled, err = btcjson.MarshalCmd(nil, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, nil, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/cmdparse.go b/btcjson/cmdparse.go index 48c6278a61..4fb8dd6260 100644 --- a/btcjson/cmdparse.go +++ b/btcjson/cmdparse.go @@ -16,26 +16,27 @@ import ( func makeParams(rt reflect.Type, rv reflect.Value) []interface{} { numFields := rt.NumField() params := make([]interface{}, 0, numFields) + lastParam := -1 for i := 0; i < numFields; i++ { rtf := rt.Field(i) rvf := rv.Field(i) + params = append(params, rvf.Interface()) if rtf.Type.Kind() == reflect.Ptr { if rvf.IsNil() { - break + // Omit optional null params unless a non-null param follows + continue } - rvf.Elem() } - params = append(params, rvf.Interface()) + lastParam = i } - - return params + return params[:lastParam+1] } // MarshalCmd marshals the passed command to a JSON-RPC request byte slice that // is suitable for transmission to an RPC server. The provided command type // must be a registered type. All commands provided by this package are // registered by default. -func MarshalCmd(id interface{}, cmd interface{}) ([]byte, error) { +func MarshalCmd(rpcVersion RPCVersion, id interface{}, cmd interface{}) ([]byte, error) { // Look up the cmd type and error out if not registered. rt := reflect.TypeOf(cmd) registerLock.RLock() @@ -59,7 +60,7 @@ func MarshalCmd(id interface{}, cmd interface{}) ([]byte, error) { params := makeParams(rt.Elem(), rv.Elem()) // Generate and marshal the final JSON-RPC request. - rawCmd, err := NewRequest(id, method, params) + rawCmd, err := NewRequest(rpcVersion, id, method, params) if err != nil { return nil, err } @@ -255,6 +256,11 @@ func assignField(paramNum int, fieldName string, dest reflect.Value, src reflect return nil } + // Optional variables can be set null using "null" string + if destIndirects > 0 && src.String() == "null" { + return nil + } + // When the destination has more indirects than the source, the extra // pointers have to be created. Only create enough pointers to reach // the same level of indirection as the source so the dest can simply be diff --git a/btcjson/cmdparse_test.go b/btcjson/cmdparse_test.go index 7c13a0bb44..f2585edf5c 100644 --- a/btcjson/cmdparse_test.go +++ b/btcjson/cmdparse_test.go @@ -162,6 +162,18 @@ func TestAssignField(t *testing.T) { src: `{"1Address":1.5}`, expected: map[string]float64{"1Address": 1.5}, }, + { + name: `null optional field - "null" -> *int32`, + dest: btcjson.Int32(0), + src: "null", + expected: nil, + }, + { + name: `null optional field - "null" -> *string`, + dest: btcjson.String(""), + src: "null", + expected: nil, + }, } t.Logf("Running %d tests", len(tests)) @@ -175,6 +187,15 @@ func TestAssignField(t *testing.T) { continue } + // Check case where null string is used on optional field + if dst.Kind() == reflect.Ptr && test.src == "null" { + if !dst.IsNil() { + t.Errorf("Test #%d (%s) unexpected value - got %v, "+ + "want nil", i, test.name, dst.Interface()) + } + continue + } + // Inidirect through to the base types to ensure their values // are the same. for dst.Kind() == reflect.Ptr { @@ -201,7 +222,7 @@ func TestAssignFieldErrors(t *testing.T) { }{ { name: "general incompatible int -> string", - dest: string(0), + dest: "\x00", src: int(0), err: btcjson.Error{ErrorCode: btcjson.ErrInvalidType}, }, @@ -401,6 +422,59 @@ func TestNewCmdErrors(t *testing.T) { } } +// TestMarshalCmd tests the MarshalCmd function. +func TestMarshalCmd(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + id interface{} + cmd interface{} + expected string + }{ + { + name: "include all parameters", + id: 1, + cmd: btcjson.NewGetNetworkHashPSCmd(btcjson.Int(100), btcjson.Int(2000)), + expected: `{"jsonrpc":"1.0","method":"getnetworkhashps","params":[100,2000],"id":1}`, + }, + { + name: "include padding null parameter", + id: 1, + cmd: btcjson.NewGetNetworkHashPSCmd(nil, btcjson.Int(2000)), + expected: `{"jsonrpc":"1.0","method":"getnetworkhashps","params":[null,2000],"id":1}`, + }, + { + name: "omit single unnecessary null parameter", + id: 1, + cmd: btcjson.NewGetNetworkHashPSCmd(btcjson.Int(100), nil), + expected: `{"jsonrpc":"1.0","method":"getnetworkhashps","params":[100],"id":1}`, + }, + { + name: "omit unnecessary null parameters", + id: 1, + cmd: btcjson.NewGetNetworkHashPSCmd(nil, nil), + expected: `{"jsonrpc":"1.0","method":"getnetworkhashps","params":[],"id":1}`, + }, + } + + t.Logf("Running %d tests", len(tests)) + for i, test := range tests { + bytes, err := btcjson.MarshalCmd(btcjson.RpcVersion1, test.id, test.cmd) + if err != nil { + t.Errorf("Test #%d (%s) wrong error - got %T (%v)", + i, test.name, err, err) + continue + } + marshalled := string(bytes) + if marshalled != test.expected { + t.Errorf("Test #%d (%s) mismatched marshall result - got "+ + "%v, want %v", i, test.name, marshalled, test.expected) + continue + } + } +} + // TestMarshalCmdErrors tests the error paths of the MarshalCmd function. func TestMarshalCmdErrors(t *testing.T) { t.Parallel() @@ -433,7 +507,7 @@ func TestMarshalCmdErrors(t *testing.T) { t.Logf("Running %d tests", len(tests)) for i, test := range tests { - _, err := btcjson.MarshalCmd(test.id, test.cmd) + _, err := btcjson.MarshalCmd(btcjson.RpcVersion1, test.id, test.cmd) if reflect.TypeOf(err) != reflect.TypeOf(test.err) { t.Errorf("Test #%d (%s) wrong error - got %T (%v), "+ "want %T", i, test.name, err, err, test.err) @@ -461,7 +535,7 @@ func TestUnmarshalCmdErrors(t *testing.T) { { name: "unregistered type", request: btcjson.Request{ - Jsonrpc: "1.0", + Jsonrpc: btcjson.RpcVersion1, Method: "bogusmethod", Params: nil, ID: nil, @@ -471,7 +545,7 @@ func TestUnmarshalCmdErrors(t *testing.T) { { name: "incorrect number of params", request: btcjson.Request{ - Jsonrpc: "1.0", + Jsonrpc: btcjson.RpcVersion1, Method: "getblockcount", Params: []json.RawMessage{[]byte(`"bogusparam"`)}, ID: nil, @@ -481,7 +555,7 @@ func TestUnmarshalCmdErrors(t *testing.T) { { name: "invalid type for a parameter", request: btcjson.Request{ - Jsonrpc: "1.0", + Jsonrpc: btcjson.RpcVersion1, Method: "getblock", Params: []json.RawMessage{[]byte("1")}, ID: nil, @@ -491,7 +565,7 @@ func TestUnmarshalCmdErrors(t *testing.T) { { name: "invalid JSON for a parameter", request: btcjson.Request{ - Jsonrpc: "1.0", + Jsonrpc: btcjson.RpcVersion1, Method: "getblock", Params: []json.RawMessage{[]byte(`"1`)}, ID: nil, diff --git a/btcjson/example_test.go b/btcjson/example_test.go index 73dd804073..74478e7482 100644 --- a/btcjson/example_test.go +++ b/btcjson/example_test.go @@ -27,7 +27,7 @@ func ExampleMarshalCmd() { // server. Typically the client would increment the id here which is // request so the response can be identified. id := 1 - marshalledBytes, err := btcjson.MarshalCmd(id, gbCmd) + marshalledBytes, err := btcjson.MarshalCmd(btcjson.RpcVersion1, id, gbCmd) if err != nil { fmt.Println(err) return @@ -95,7 +95,7 @@ func ExampleUnmarshalCmd() { func ExampleMarshalResponse() { // Marshal a new JSON-RPC response. For example, this is a response // to a getblockheight request. - marshalledBytes, err := btcjson.MarshalResponse(1, 350001, nil) + marshalledBytes, err := btcjson.MarshalResponse(btcjson.RpcVersion1, 1, 350001, nil) if err != nil { fmt.Println(err) return @@ -107,7 +107,7 @@ func ExampleMarshalResponse() { fmt.Printf("%s\n", marshalledBytes) // Output: - // {"result":350001,"error":null,"id":1} + // {"jsonrpc":"1.0","result":350001,"error":null,"id":1} } // This example demonstrates how to unmarshal a JSON-RPC response and then @@ -116,7 +116,7 @@ func Example_unmarshalResponse() { // Ordinarily this would be read from the wire, but for this example, // it is hard coded here for clarity. This is an example response to a // getblockheight request. - data := []byte(`{"result":350001,"error":null,"id":1}`) + data := []byte(`{"jsonrpc":"1.0","result":350001,"error":null,"id":1}`) // Unmarshal the raw bytes from the wire into a JSON-RPC response. var response btcjson.Response diff --git a/btcjson/helpers.go b/btcjson/helpers.go index d9b452e7c3..eda26cb885 100644 --- a/btcjson/helpers.go +++ b/btcjson/helpers.go @@ -75,3 +75,11 @@ func String(v string) *string { *p = v return p } + +// NewFilterTypeName is a helper routine that allocates a new FilterTypeName value to store v and +// returns a pointer to it. This is useful when assigning optional parameters. +func NewFilterTypeName(v FilterTypeName) *FilterTypeName { + p := new(FilterTypeName) + *p = v + return p +} diff --git a/btcjson/jsonrpc.go b/btcjson/jsonrpc.go index 0ead85e5ee..553a7bc37f 100644 --- a/btcjson/jsonrpc.go +++ b/btcjson/jsonrpc.go @@ -9,6 +9,33 @@ import ( "fmt" ) +// RPCVersion is a type to indicate RPC versions. +type RPCVersion string + +const ( + // version 1 of rpc + RpcVersion1 RPCVersion = RPCVersion("1.0") + // version 2 of rpc + RpcVersion2 RPCVersion = RPCVersion("2.0") +) + +var validRpcVersions = []RPCVersion{RpcVersion1, RpcVersion2} + +// check if the rpc version is a valid version +func (r RPCVersion) IsValid() bool { + for _, version := range validRpcVersions { + if version == r { + return true + } + } + return false +} + +// cast rpc version to a string +func (r RPCVersion) String() string { + return string(r) +} + // RPCErrorCode represents an error code to be used as a part of an RPCError // which is in turn used in a JSON-RPC Response object. // @@ -67,21 +94,74 @@ func IsValidIDType(id interface{}) bool { // requests, however this struct it being exported in case the caller wants to // construct raw requests for some reason. type Request struct { - Jsonrpc string `json:"jsonrpc"` + Jsonrpc RPCVersion `json:"jsonrpc"` Method string `json:"method"` Params []json.RawMessage `json:"params"` ID interface{} `json:"id"` } -// NewRequest returns a new JSON-RPC 1.0 request object given the provided id, -// method, and parameters. The parameters are marshalled into a json.RawMessage -// for the Params field of the returned request object. This function is only -// provided in case the caller wants to construct raw requests for some reason. -// -// Typically callers will instead want to create a registered concrete command -// type with the NewCmd or NewCmd functions and call the MarshalCmd -// function with that command to generate the marshalled JSON-RPC request. -func NewRequest(id interface{}, method string, params []interface{}) (*Request, error) { +// UnmarshalJSON is a custom unmarshal func for the Request struct. The param +// field defaults to an empty json.RawMessage array it is omitted by the request +// or nil if the supplied value is invalid. +func (request *Request) UnmarshalJSON(b []byte) error { + // Step 1: Create a type alias of the original struct. + type Alias Request + + // Step 2: Create an anonymous struct with raw replacements for the special + // fields. + aux := &struct { + Jsonrpc string `json:"jsonrpc"` + Params []interface{} `json:"params"` + *Alias + }{ + Alias: (*Alias)(request), + } + + // Step 3: Unmarshal the data into the anonymous struct. + err := json.Unmarshal(b, &aux) + if err != nil { + return err + } + + // Step 4: Convert the raw fields to the desired types + + version := RPCVersion(aux.Jsonrpc) + if version.IsValid() { + request.Jsonrpc = version + } + + rawParams := make([]json.RawMessage, 0) + + for _, param := range aux.Params { + marshalledParam, err := json.Marshal(param) + if err != nil { + return err + } + + rawMessage := json.RawMessage(marshalledParam) + rawParams = append(rawParams, rawMessage) + } + + request.Params = rawParams + + return nil +} + +// NewRequest returns a new JSON-RPC request object given the provided rpc +// version, id, method, and parameters. The parameters are marshalled into a +// json.RawMessage for the Params field of the returned request object. This +// function is only provided in case the caller wants to construct raw requests +// for some reason. Typically callers will instead want to create a registered +// concrete command type with the NewCmd or NewCmd functions and call the +// MarshalCmd function with that command to generate the marshalled JSON-RPC +// request. +func NewRequest(rpcVersion RPCVersion, id interface{}, method string, params []interface{}) (*Request, error) { + // default to JSON-RPC 1.0 if RPC type is not specified + if !rpcVersion.IsValid() { + str := fmt.Sprintf("rpcversion '%s' is invalid", rpcVersion) + return nil, makeError(ErrInvalidType, str) + } + if !IsValidIDType(id) { str := fmt.Sprintf("the id of type '%T' is invalid", id) return nil, makeError(ErrInvalidType, str) @@ -98,30 +178,35 @@ func NewRequest(id interface{}, method string, params []interface{}) (*Request, } return &Request{ - Jsonrpc: "1.0", + Jsonrpc: rpcVersion, ID: id, Method: method, Params: rawParams, }, nil } -// Response is the general form of a JSON-RPC response. The type of the Result -// field varies from one command to the next, so it is implemented as an -// interface. The ID field has to be a pointer for Go to put a null in it when +// Response is the general form of a JSON-RPC response. The type of the +// Result field varies from one command to the next, so it is implemented as an +// interface. The ID field has to be a pointer to allow for a nil value when // empty. type Response struct { - Result json.RawMessage `json:"result"` - Error *RPCError `json:"error"` - ID *interface{} `json:"id"` + Jsonrpc RPCVersion `json:"jsonrpc"` + Result json.RawMessage `json:"result"` + Error *RPCError `json:"error"` + ID *interface{} `json:"id"` } -// NewResponse returns a new JSON-RPC response object given the provided id, -// marshalled result, and RPC error. This function is only provided in case the -// caller wants to construct raw responses for some reason. -// +// NewResponse returns a new JSON-RPC response object given the provided rpc +// version, id, marshalled result, and RPC error. This function is only +// provided in case the caller wants to construct raw responses for some reason. // Typically callers will instead want to create the fully marshalled JSON-RPC // response to send over the wire with the MarshalResponse function. -func NewResponse(id interface{}, marshalledResult []byte, rpcErr *RPCError) (*Response, error) { +func NewResponse(rpcVersion RPCVersion, id interface{}, marshalledResult []byte, rpcErr *RPCError) (*Response, error) { + if !rpcVersion.IsValid() { + str := fmt.Sprintf("rpcversion '%s' is invalid", rpcVersion) + return nil, makeError(ErrInvalidType, str) + } + if !IsValidIDType(id) { str := fmt.Sprintf("the id of type '%T' is invalid", id) return nil, makeError(ErrInvalidType, str) @@ -129,20 +214,27 @@ func NewResponse(id interface{}, marshalledResult []byte, rpcErr *RPCError) (*Re pid := &id return &Response{ - Result: marshalledResult, - Error: rpcErr, - ID: pid, + Jsonrpc: rpcVersion, + Result: marshalledResult, + Error: rpcErr, + ID: pid, }, nil } -// MarshalResponse marshals the passed id, result, and RPCError to a JSON-RPC -// response byte slice that is suitable for transmission to a JSON-RPC client. -func MarshalResponse(id interface{}, result interface{}, rpcErr *RPCError) ([]byte, error) { +// MarshalResponse marshals the passed rpc version, id, result, and RPCError to +// a JSON-RPC response byte slice that is suitable for transmission to a +// JSON-RPC client. +func MarshalResponse(rpcVersion RPCVersion, id interface{}, result interface{}, rpcErr *RPCError) ([]byte, error) { + if !rpcVersion.IsValid() { + str := fmt.Sprintf("rpcversion '%s' is invalid", rpcVersion) + return nil, makeError(ErrInvalidType, str) + } + marshalledResult, err := json.Marshal(result) if err != nil { return nil, err } - response, err := NewResponse(id, marshalledResult, rpcErr) + response, err := NewResponse(rpcVersion, id, marshalledResult, rpcErr) if err != nil { return nil, err } diff --git a/btcjson/jsonrpc_test.go b/btcjson/jsonrpc_test.go index 7a5d75618c..13d98e89f5 100644 --- a/btcjson/jsonrpc_test.go +++ b/btcjson/jsonrpc_test.go @@ -68,7 +68,7 @@ func TestMarshalResponse(t *testing.T) { name: "ordinary bool result with no error", result: true, jsonErr: nil, - expected: []byte(`{"result":true,"error":null,"id":1}`), + expected: []byte(`{"jsonrpc":"1.0","result":true,"error":null,"id":1}`), }, { name: "result with error", @@ -76,14 +76,14 @@ func TestMarshalResponse(t *testing.T) { jsonErr: func() *btcjson.RPCError { return btcjson.NewRPCError(btcjson.ErrRPCBlockNotFound, "123 not found") }(), - expected: []byte(`{"result":null,"error":{"code":-5,"message":"123 not found"},"id":1}`), + expected: []byte(`{"jsonrpc":"1.0","result":null,"error":{"code":-5,"message":"123 not found"},"id":1}`), }, } t.Logf("Running %d tests", len(tests)) for i, test := range tests { _, _ = i, test - marshalled, err := btcjson.MarshalResponse(testID, test.result, test.jsonErr) + marshalled, err := btcjson.MarshalResponse(btcjson.RpcVersion1, testID, test.result, test.jsonErr) if err != nil { t.Errorf("Test #%d (%s) unexpected error: %v", i, test.name, err) @@ -104,7 +104,7 @@ func TestMiscErrors(t *testing.T) { // Force an error in NewRequest by giving it a parameter type that is // not supported. - _, err := btcjson.NewRequest(nil, "test", []interface{}{make(chan int)}) + _, err := btcjson.NewRequest(btcjson.RpcVersion1, nil, "test", []interface{}{make(chan int)}) if err == nil { t.Error("NewRequest: did not receive error") return @@ -113,7 +113,7 @@ func TestMiscErrors(t *testing.T) { // Force an error in MarshalResponse by giving it an id type that is not // supported. wantErr := btcjson.Error{ErrorCode: btcjson.ErrInvalidType} - _, err = btcjson.MarshalResponse(make(chan int), nil, nil) + _, err = btcjson.MarshalResponse(btcjson.RpcVersion1, make(chan int), nil, nil) if jerr, ok := err.(btcjson.Error); !ok || jerr.ErrorCode != wantErr.ErrorCode { t.Errorf("MarshalResult: did not receive expected error - got "+ "%v (%[1]T), want %v (%[2]T)", err, wantErr) @@ -122,7 +122,7 @@ func TestMiscErrors(t *testing.T) { // Force an error in MarshalResponse by giving it a result type that // can't be marshalled. - _, err = btcjson.MarshalResponse(1, make(chan int), nil) + _, err = btcjson.MarshalResponse(btcjson.RpcVersion1, 1, make(chan int), nil) if _, ok := err.(*json.UnsupportedTypeError); !ok { wantErr := &json.UnsupportedTypeError{} t.Errorf("MarshalResult: did not receive expected error - got "+ diff --git a/btcjson/jsonrpcerr.go b/btcjson/jsonrpcerr.go index ea62fb55a6..d67b58bbb6 100644 --- a/btcjson/jsonrpcerr.go +++ b/btcjson/jsonrpcerr.go @@ -30,36 +30,133 @@ var ( // General application defined JSON errors. const ( - ErrRPCMisc RPCErrorCode = -1 + // ErrRPCMisc indicates an exception thrown during command handling. + ErrRPCMisc RPCErrorCode = -1 + + // ErrRPCForbiddenBySafeMode indicates that server is in safe mode, and + // command is not allowed in safe mode. ErrRPCForbiddenBySafeMode RPCErrorCode = -2 - ErrRPCType RPCErrorCode = -3 + + // ErrRPCType indicates that an unexpected type was passed as parameter. + ErrRPCType RPCErrorCode = -3 + + // ErrRPCInvalidAddressOrKey indicates an invalid address or key. ErrRPCInvalidAddressOrKey RPCErrorCode = -5 - ErrRPCOutOfMemory RPCErrorCode = -7 - ErrRPCInvalidParameter RPCErrorCode = -8 - ErrRPCDatabase RPCErrorCode = -20 - ErrRPCDeserialization RPCErrorCode = -22 - ErrRPCVerify RPCErrorCode = -25 - ErrRPCInWarmup RPCErrorCode = -28 + + // ErrRPCOutOfMemory indicates that the server ran out of memory during + // operation. + ErrRPCOutOfMemory RPCErrorCode = -7 + + // ErrRPCInvalidParameter indicates an invalid, missing, or duplicate + // parameter. + ErrRPCInvalidParameter RPCErrorCode = -8 + + // ErrRPCDatabase indicates a database error. + ErrRPCDatabase RPCErrorCode = -20 + + // ErrRPCDeserialization indicates an error parsing or validating structure + // in raw format. + ErrRPCDeserialization RPCErrorCode = -22 + + // ErrRPCVerify indicates a general error during transaction or block + // submission. + ErrRPCVerify RPCErrorCode = -25 + + // ErrRPCVerifyRejected indicates that transaction or block was rejected by + // network rules. + ErrRPCVerifyRejected RPCErrorCode = -26 + + // ErrRPCVerifyAlreadyInChain indicates that submitted transaction is + // already in chain. + ErrRPCVerifyAlreadyInChain RPCErrorCode = -27 + + // ErrRPCInWarmup indicates that client is still warming up. + ErrRPCInWarmup RPCErrorCode = -28 + + // ErrRPCInWarmup indicates that the RPC error is deprecated. + ErrRPCMethodDeprecated RPCErrorCode = -32 ) // Peer-to-peer client errors. const ( - ErrRPCClientNotConnected RPCErrorCode = -9 + // ErrRPCClientNotConnected indicates that Bitcoin is not connected. + ErrRPCClientNotConnected RPCErrorCode = -9 + + // ErrRPCClientInInitialDownload indicates that client is still downloading + // initial blocks. ErrRPCClientInInitialDownload RPCErrorCode = -10 - ErrRPCClientNodeNotAdded RPCErrorCode = -24 + + // ErrRPCClientNodeAlreadyAdded indicates that node is already added. + ErrRPCClientNodeAlreadyAdded RPCErrorCode = -23 + + // ErrRPCClientNodeNotAdded indicates that node has not been added before. + ErrRPCClientNodeNotAdded RPCErrorCode = -24 + + // ErrRPCClientNodeNotConnected indicates that node to disconnect was not + // found in connected nodes. + ErrRPCClientNodeNotConnected RPCErrorCode = -29 + + // ErrRPCClientInvalidIPOrSubnet indicates an invalid IP/Subnet. + ErrRPCClientInvalidIPOrSubnet RPCErrorCode = -30 + + // ErrRPCClientP2PDisabled indicates that no valid connection manager + // instance was found. + ErrRPCClientP2PDisabled RPCErrorCode = -31 +) + +// Chain errors +const ( + // ErrRPCClientMempoolDisabled indicates that no mempool instance was + // found. + ErrRPCClientMempoolDisabled RPCErrorCode = -33 ) // Wallet JSON errors const ( - ErrRPCWallet RPCErrorCode = -4 - ErrRPCWalletInsufficientFunds RPCErrorCode = -6 - ErrRPCWalletInvalidAccountName RPCErrorCode = -11 - ErrRPCWalletKeypoolRanOut RPCErrorCode = -12 - ErrRPCWalletUnlockNeeded RPCErrorCode = -13 + // ErrRPCWallet indicates an unspecified problem with wallet, for + // example, key not found, etc. + ErrRPCWallet RPCErrorCode = -4 + + // ErrRPCWalletInsufficientFunds indicates that there are not enough + // funds in wallet or account. + ErrRPCWalletInsufficientFunds RPCErrorCode = -6 + + // ErrRPCWalletInvalidAccountName indicates an invalid label name. + ErrRPCWalletInvalidAccountName RPCErrorCode = -11 + + // ErrRPCWalletKeypoolRanOut indicates that the keypool ran out, and that + // keypoolrefill must be called first. + ErrRPCWalletKeypoolRanOut RPCErrorCode = -12 + + // ErrRPCWalletUnlockNeeded indicates that the wallet passphrase must be + // entered first with the walletpassphrase RPC. + ErrRPCWalletUnlockNeeded RPCErrorCode = -13 + + // ErrRPCWalletPassphraseIncorrect indicates that the wallet passphrase + // that was entered was incorrect. ErrRPCWalletPassphraseIncorrect RPCErrorCode = -14 - ErrRPCWalletWrongEncState RPCErrorCode = -15 - ErrRPCWalletEncryptionFailed RPCErrorCode = -16 - ErrRPCWalletAlreadyUnlocked RPCErrorCode = -17 + + // ErrRPCWalletWrongEncState indicates that a command was given in wrong + // wallet encryption state, for example, encrypting an encrypted wallet. + ErrRPCWalletWrongEncState RPCErrorCode = -15 + + // ErrRPCWalletEncryptionFailed indicates a failure to encrypt the wallet. + ErrRPCWalletEncryptionFailed RPCErrorCode = -16 + + // ErrRPCWalletAlreadyUnlocked indicates an attempt to unlock a wallet + // that was already unlocked. + ErrRPCWalletAlreadyUnlocked RPCErrorCode = -17 + + // ErrRPCWalletNotFound indicates that an invalid wallet was specified, + // which does not exist. It can also indicate an attempt to unload a + // wallet that was not previously loaded. + // + // Not to be confused with ErrRPCNoWallet, which is specific to btcd. + ErrRPCWalletNotFound RPCErrorCode = -18 + + // ErrRPCWalletNotSpecified indicates that no wallet was specified, for + // example, when there are multiple wallets loaded. + ErrRPCWalletNotSpecified RPCErrorCode = -19 ) // Specific Errors related to commands. These are the ones a user of the RPC diff --git a/btcjson/walletsvrcmds.go b/btcjson/walletsvrcmds.go index be1a67fbf0..2ff9ae1832 100644 --- a/btcjson/walletsvrcmds.go +++ b/btcjson/walletsvrcmds.go @@ -1,4 +1,4 @@ -// Copyright (c) 2014 The btcsuite developers +// Copyright (c) 2014-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. @@ -7,6 +7,14 @@ package btcjson +import ( + "encoding/hex" + "encoding/json" + "fmt" + + "github.com/btcsuite/btcutil" +) + // AddMultisigAddressCmd defines the addmutisigaddress JSON-RPC command. type AddMultisigAddressCmd struct { NRequired int @@ -55,6 +63,28 @@ func NewCreateMultisigCmd(nRequired int, keys []string) *CreateMultisigCmd { } } +// CreateWalletCmd defines the createwallet JSON-RPC command. +type CreateWalletCmd struct { + WalletName string + DisablePrivateKeys *bool `jsonrpcdefault:"false"` + Blank *bool `jsonrpcdefault:"false"` + Passphrase *string `jsonrpcdefault:"\"\""` + AvoidReuse *bool `jsonrpcdefault:"false"` +} + +// NewCreateWalletCmd returns a new instance which can be used to issue a +// createwallet JSON-RPC command. +func NewCreateWalletCmd(walletName string, disablePrivateKeys *bool, + blank *bool, passphrase *string, avoidReuse *bool) *CreateWalletCmd { + return &CreateWalletCmd{ + WalletName: walletName, + DisablePrivateKeys: disablePrivateKeys, + Blank: blank, + Passphrase: passphrase, + AvoidReuse: avoidReuse, + } +} + // DumpPrivKeyCmd defines the dumpprivkey JSON-RPC command. type DumpPrivKeyCmd struct { Address string @@ -170,6 +200,19 @@ func NewGetAddressesByAccountCmd(account string) *GetAddressesByAccountCmd { } } +// GetAddressInfoCmd defines the getaddressinfo JSON-RPC command. +type GetAddressInfoCmd struct { + Address string +} + +// NewGetAddressInfoCmd returns a new instance which can be used to issue a +// getaddressinfo JSON-RPC command. +func NewGetAddressInfoCmd(address string) *GetAddressInfoCmd { + return &GetAddressInfoCmd{ + Address: address, + } +} + // GetBalanceCmd defines the getbalance JSON-RPC command. type GetBalanceCmd struct { Account *string @@ -292,6 +335,39 @@ func NewGetWalletInfoCmd() *GetWalletInfoCmd { return &GetWalletInfoCmd{} } +// BackupWalletCmd defines the backupwallet JSON-RPC command +type BackupWalletCmd struct { + Destination string +} + +// NewBackupWalletCmd returns a new instance which can be used to issue a +// backupwallet JSON-RPC command +func NewBackupWalletCmd(destination string) *BackupWalletCmd { + return &BackupWalletCmd{Destination: destination} +} + +// UnloadWalletCmd defines the unloadwallet JSON-RPC command +type UnloadWalletCmd struct { + WalletName *string +} + +// NewUnloadWalletCmd returns a new instance which can be used to issue a +// unloadwallet JSON-RPC command. +func NewUnloadWalletCmd(walletName *string) *UnloadWalletCmd { + return &UnloadWalletCmd{WalletName: walletName} +} + +// LoadWalletCmd defines the loadwallet JSON-RPC command +type LoadWalletCmd struct { + WalletName string +} + +// NewLoadWalletCmd returns a new instance which can be used to issue a +// loadwallet JSON-RPC command +func NewLoadWalletCmd(walletName string) *LoadWalletCmd { + return &LoadWalletCmd{WalletName: walletName} +} + // ImportPrivKeyCmd defines the importprivkey JSON-RPC command. type ImportPrivKeyCmd struct { PrivKey string @@ -647,6 +723,39 @@ func NewSignRawTransactionCmd(hexEncodedTx string, inputs *[]RawTxInput, privKey } } +// RawTxWitnessInput models the data needed for raw transaction input that is used in +// the SignRawTransactionWithWalletCmd struct. The RedeemScript is required for P2SH inputs, +// the WitnessScript is required for P2WSH or P2SH-P2WSH witness scripts, and the Amount is +// required for Segwit inputs. Otherwise, those fields can be left blank. +type RawTxWitnessInput struct { + Txid string `json:"txid"` + Vout uint32 `json:"vout"` + ScriptPubKey string `json:"scriptPubKey"` + RedeemScript *string `json:"redeemScript,omitempty"` + WitnessScript *string `json:"witnessScript,omitempty"` + Amount *float64 `json:"amount,omitempty"` // In BTC +} + +// SignRawTransactionWithWalletCmd defines the signrawtransactionwithwallet JSON-RPC command. +type SignRawTransactionWithWalletCmd struct { + RawTx string + Inputs *[]RawTxWitnessInput + SigHashType *string `jsonrpcdefault:"\"ALL\""` +} + +// NewSignRawTransactionWithWalletCmd returns a new instance which can be used to issue a +// signrawtransactionwithwallet JSON-RPC command. +// +// The parameters which are pointers indicate they are optional. Passing nil +// for optional parameters will use the default value. +func NewSignRawTransactionWithWalletCmd(hexEncodedTx string, inputs *[]RawTxWitnessInput, sigHashType *string) *SignRawTransactionWithWalletCmd { + return &SignRawTransactionWithWalletCmd{ + RawTx: hexEncodedTx, + Inputs: inputs, + SigHashType: sigHashType, + } +} + // WalletLockCmd defines the walletlock JSON-RPC command. type WalletLockCmd struct{} @@ -686,13 +795,299 @@ func NewWalletPassphraseChangeCmd(oldPassphrase, newPassphrase string) *WalletPa } } +// TimestampOrNow defines a type to represent a timestamp value in seconds, +// since epoch. +// +// The value can either be a integer, or the string "now". +// +// NOTE: Interpretation of the timestamp value depends upon the specific +// JSON-RPC command, where it is used. +type TimestampOrNow struct { + Value interface{} +} + +// MarshalJSON implements the json.Marshaler interface for TimestampOrNow +func (t TimestampOrNow) MarshalJSON() ([]byte, error) { + return json.Marshal(t.Value) +} + +// UnmarshalJSON implements the json.Unmarshaler interface for TimestampOrNow +func (t *TimestampOrNow) UnmarshalJSON(data []byte) error { + var unmarshalled interface{} + if err := json.Unmarshal(data, &unmarshalled); err != nil { + return err + } + + switch v := unmarshalled.(type) { + case float64: + t.Value = int(v) + case string: + if v != "now" { + return fmt.Errorf("invalid timestamp value: %v", unmarshalled) + } + t.Value = v + default: + return fmt.Errorf("invalid timestamp value: %v", unmarshalled) + } + return nil +} + +// ScriptPubKeyAddress represents an address, to be used in conjunction with +// ScriptPubKey. +type ScriptPubKeyAddress struct { + Address string `json:"address"` +} + +// ScriptPubKey represents a script (as a string) or an address +// (as a ScriptPubKeyAddress). +type ScriptPubKey struct { + Value interface{} +} + +// MarshalJSON implements the json.Marshaler interface for ScriptPubKey +func (s ScriptPubKey) MarshalJSON() ([]byte, error) { + return json.Marshal(s.Value) +} + +// UnmarshalJSON implements the json.Unmarshaler interface for ScriptPubKey +func (s *ScriptPubKey) UnmarshalJSON(data []byte) error { + var unmarshalled interface{} + if err := json.Unmarshal(data, &unmarshalled); err != nil { + return err + } + + switch v := unmarshalled.(type) { + case string: + s.Value = v + case map[string]interface{}: + s.Value = ScriptPubKeyAddress{Address: v["address"].(string)} + default: + return fmt.Errorf("invalid scriptPubKey value: %v", unmarshalled) + } + return nil +} + +// DescriptorRange specifies the limits of a ranged Descriptor. +// +// Descriptors are typically ranged when specified in the form of generic HD +// chain paths. +// Example of a ranged descriptor: pkh(tpub.../*) +// +// The value can be an int to specify the end of the range, or the range +// itself, as []int{begin, end}. +type DescriptorRange struct { + Value interface{} +} + +// MarshalJSON implements the json.Marshaler interface for DescriptorRange +func (r DescriptorRange) MarshalJSON() ([]byte, error) { + return json.Marshal(r.Value) +} + +// UnmarshalJSON implements the json.Unmarshaler interface for DescriptorRange +func (r *DescriptorRange) UnmarshalJSON(data []byte) error { + var unmarshalled interface{} + if err := json.Unmarshal(data, &unmarshalled); err != nil { + return err + } + + switch v := unmarshalled.(type) { + case float64: + r.Value = int(v) + case []interface{}: + if len(v) != 2 { + return fmt.Errorf("expected [begin,end] integer range, got: %v", unmarshalled) + } + r.Value = []int{ + int(v[0].(float64)), + int(v[1].(float64)), + } + default: + return fmt.Errorf("invalid descriptor range value: %v", unmarshalled) + } + return nil +} + +// ImportMultiRequest defines the request struct to be passed to the +// ImportMultiCmd, as an array. +type ImportMultiRequest struct { + // Descriptor to import, in canonical form. If using Descriptor, do not + // also provide ScriptPubKey, RedeemScript, WitnessScript, PubKeys, or Keys. + Descriptor *string `json:"desc,omitempty"` + + // Script/address to import. Should not be provided if using Descriptor. + ScriptPubKey *ScriptPubKey `json:"scriptPubKey,omitempty"` + + // Creation time of the key in seconds since epoch (Jan 1 1970 GMT), or + // the string "now" to substitute the current synced blockchain time. + // + // The timestamp of the oldest key will determine how far back blockchain + // rescans need to begin for missing wallet transactions. + // + // Specifying "now" bypasses scanning. Useful for keys that are known to + // never have been used. + // + // Specifying 0 scans the entire blockchain. + Timestamp TimestampOrNow `json:"timestamp"` + + // Allowed only if the ScriptPubKey is a P2SH or P2SH-P2WSH + // address/scriptPubKey. + RedeemScript *string `json:"redeemscript,omitempty"` + + // Allowed only if the ScriptPubKey is a P2SH-P2WSH or P2WSH + // address/scriptPubKey. + WitnessScript *string `json:"witnessscript,omitempty"` + + // Array of strings giving pubkeys to import. They must occur in P2PKH or + // P2WPKH scripts. They are not required when the private key is also + // provided (see Keys). + PubKeys *[]string `json:"pubkeys,omitempty"` + + // Array of strings giving private keys to import. The corresponding + // public keys must occur in the output or RedeemScript. + Keys *[]string `json:"keys,omitempty"` + + // If the provided Descriptor is ranged, this specifies the end + // (as an int) or the range (as []int{begin, end}) to import. + Range *DescriptorRange `json:"range,omitempty"` + + // States whether matching outputs should be treated as not incoming + // payments (also known as change). + Internal *bool `json:"internal,omitempty"` + + // States whether matching outputs should be considered watchonly. + // + // If an address/script is imported without all of the private keys + // required to spend from that address, set this field to true. + // + // If all the private keys are provided and the address/script is + // spendable, set this field to false. + WatchOnly *bool `json:"watchonly,omitempty"` + + // Label to assign to the address. Only allowed when Internal is false. + Label *string `json:"label,omitempty"` + + // States whether imported public keys should be added to the keypool for + // when users request new addresses. Only allowed when wallet private keys + // are disabled. + KeyPool *bool `json:"keypool,omitempty"` +} + +// ImportMultiRequest defines the options struct, provided to the +// ImportMultiCmd as a pointer argument. +type ImportMultiOptions struct { + Rescan bool `json:"rescan"` // Rescan the blockchain after all imports +} + +// ImportMultiCmd defines the importmulti JSON-RPC command. +type ImportMultiCmd struct { + Requests []ImportMultiRequest + Options *ImportMultiOptions +} + +// NewImportMultiCmd returns a new instance which can be used to issue +// an importmulti JSON-RPC command. +// +// The parameters which are pointers indicate they are optional. Passing nil +// for optional parameters will use the default value. +func NewImportMultiCmd(requests []ImportMultiRequest, options *ImportMultiOptions) *ImportMultiCmd { + return &ImportMultiCmd{ + Requests: requests, + Options: options, + } +} + +// PsbtInput represents an input to include in the PSBT created by the +// WalletCreateFundedPsbtCmd command. +type PsbtInput struct { + Txid string `json:"txid"` + Vout uint32 `json:"vout"` + Sequence uint32 `json:"sequence"` +} + +// PsbtOutput represents an output to include in the PSBT created by the +// WalletCreateFundedPsbtCmd command. +type PsbtOutput map[string]interface{} + +// NewPsbtOutput returns a new instance of a PSBT output to use with the +// WalletCreateFundedPsbtCmd command. +func NewPsbtOutput(address string, amount btcutil.Amount) PsbtOutput { + return PsbtOutput{address: amount.ToBTC()} +} + +// NewPsbtDataOutput returns a new instance of a PSBT data output to use with +// the WalletCreateFundedPsbtCmd command. +func NewPsbtDataOutput(data []byte) PsbtOutput { + return PsbtOutput{"data": hex.EncodeToString(data)} +} + +// WalletCreateFundedPsbtOpts represents the optional options struct provided +// with a WalletCreateFundedPsbtCmd command. +type WalletCreateFundedPsbtOpts struct { + ChangeAddress *string `json:"changeAddress,omitempty"` + ChangePosition *int64 `json:"changePosition,omitempty"` + ChangeType *ChangeType `json:"change_type,omitempty"` + IncludeWatching *bool `json:"includeWatching,omitempty"` + LockUnspents *bool `json:"lockUnspents,omitempty"` + FeeRate *int64 `json:"feeRate,omitempty"` + SubtractFeeFromOutputs *[]int64 `json:"subtractFeeFromOutputs,omitempty"` + Replaceable *bool `json:"replaceable,omitempty"` + ConfTarget *int64 `json:"conf_target,omitempty"` + EstimateMode *string `json:"estimate_mode,omitempty"` +} + +// WalletCreateFundedPsbtCmd defines the walletcreatefundedpsbt JSON-RPC command. +type WalletCreateFundedPsbtCmd struct { + Inputs []PsbtInput + Outputs []PsbtOutput + Locktime *uint32 + Options *WalletCreateFundedPsbtOpts + Bip32Derivs *bool +} + +// NewWalletCreateFundedPsbtCmd returns a new instance which can be used to issue a +// walletcreatefundedpsbt JSON-RPC command. +func NewWalletCreateFundedPsbtCmd( + inputs []PsbtInput, outputs []PsbtOutput, locktime *uint32, + options *WalletCreateFundedPsbtOpts, bip32Derivs *bool, +) *WalletCreateFundedPsbtCmd { + return &WalletCreateFundedPsbtCmd{ + Inputs: inputs, + Outputs: outputs, + Locktime: locktime, + Options: options, + Bip32Derivs: bip32Derivs, + } +} + +// WalletProcessPsbtCmd defines the walletprocesspsbt JSON-RPC command. +type WalletProcessPsbtCmd struct { + Psbt string + Sign *bool `jsonrpcdefault:"true"` + SighashType *string `jsonrpcdefault:"\"ALL\""` + Bip32Derivs *bool +} + +// NewWalletProcessPsbtCmd returns a new instance which can be used to issue a +// walletprocesspsbt JSON-RPC command. +func NewWalletProcessPsbtCmd(psbt string, sign *bool, sighashType *string, bip32Derivs *bool) *WalletProcessPsbtCmd { + return &WalletProcessPsbtCmd{ + Psbt: psbt, + Sign: sign, + SighashType: sighashType, + Bip32Derivs: bip32Derivs, + } +} + func init() { // The commands in this file are only usable with a wallet server. flags := UFWalletOnly MustRegisterCmd("addmultisigaddress", (*AddMultisigAddressCmd)(nil), flags) MustRegisterCmd("addwitnessaddress", (*AddWitnessAddressCmd)(nil), flags) + MustRegisterCmd("backupwallet", (*BackupWalletCmd)(nil), flags) MustRegisterCmd("createmultisig", (*CreateMultisigCmd)(nil), flags) + MustRegisterCmd("createwallet", (*CreateWalletCmd)(nil), flags) MustRegisterCmd("dumpprivkey", (*DumpPrivKeyCmd)(nil), flags) MustRegisterCmd("encryptwallet", (*EncryptWalletCmd)(nil), flags) MustRegisterCmd("estimatesmartfee", (*EstimateSmartFeeCmd)(nil), flags) @@ -701,6 +1096,7 @@ func init() { MustRegisterCmd("getaccount", (*GetAccountCmd)(nil), flags) MustRegisterCmd("getaccountaddress", (*GetAccountAddressCmd)(nil), flags) MustRegisterCmd("getaddressesbyaccount", (*GetAddressesByAccountCmd)(nil), flags) + MustRegisterCmd("getaddressinfo", (*GetAddressInfoCmd)(nil), flags) MustRegisterCmd("getbalance", (*GetBalanceCmd)(nil), flags) MustRegisterCmd("getbalances", (*GetBalancesCmd)(nil), flags) MustRegisterCmd("getnewaddress", (*GetNewAddressCmd)(nil), flags) @@ -709,6 +1105,7 @@ func init() { MustRegisterCmd("getreceivedbyaddress", (*GetReceivedByAddressCmd)(nil), flags) MustRegisterCmd("gettransaction", (*GetTransactionCmd)(nil), flags) MustRegisterCmd("getwalletinfo", (*GetWalletInfoCmd)(nil), flags) + MustRegisterCmd("importmulti", (*ImportMultiCmd)(nil), flags) MustRegisterCmd("importprivkey", (*ImportPrivKeyCmd)(nil), flags) MustRegisterCmd("keypoolrefill", (*KeyPoolRefillCmd)(nil), flags) MustRegisterCmd("listaccounts", (*ListAccountsCmd)(nil), flags) @@ -719,6 +1116,7 @@ func init() { MustRegisterCmd("listsinceblock", (*ListSinceBlockCmd)(nil), flags) MustRegisterCmd("listtransactions", (*ListTransactionsCmd)(nil), flags) MustRegisterCmd("listunspent", (*ListUnspentCmd)(nil), flags) + MustRegisterCmd("loadwallet", (*LoadWalletCmd)(nil), flags) MustRegisterCmd("lockunspent", (*LockUnspentCmd)(nil), flags) MustRegisterCmd("move", (*MoveCmd)(nil), flags) MustRegisterCmd("sendfrom", (*SendFromCmd)(nil), flags) @@ -728,7 +1126,11 @@ func init() { MustRegisterCmd("settxfee", (*SetTxFeeCmd)(nil), flags) MustRegisterCmd("signmessage", (*SignMessageCmd)(nil), flags) MustRegisterCmd("signrawtransaction", (*SignRawTransactionCmd)(nil), flags) + MustRegisterCmd("signrawtransactionwithwallet", (*SignRawTransactionWithWalletCmd)(nil), flags) + MustRegisterCmd("unloadwallet", (*UnloadWalletCmd)(nil), flags) MustRegisterCmd("walletlock", (*WalletLockCmd)(nil), flags) MustRegisterCmd("walletpassphrase", (*WalletPassphraseCmd)(nil), flags) MustRegisterCmd("walletpassphrasechange", (*WalletPassphraseChangeCmd)(nil), flags) + MustRegisterCmd("walletcreatefundedpsbt", (*WalletCreateFundedPsbtCmd)(nil), flags) + MustRegisterCmd("walletprocesspsbt", (*WalletProcessPsbtCmd)(nil), flags) } diff --git a/btcjson/walletsvrcmds_test.go b/btcjson/walletsvrcmds_test.go index 554a87413a..9c68d260f6 100644 --- a/btcjson/walletsvrcmds_test.go +++ b/btcjson/walletsvrcmds_test.go @@ -1,4 +1,4 @@ -// Copyright (c) 2014 The btcsuite developers +// Copyright (c) 2014-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. @@ -12,6 +12,7 @@ import ( "testing" "github.com/btcsuite/btcd/btcjson" + "github.com/btcsuite/btcutil" ) // TestWalletSvrCmds tests all of the wallet server commands marshal and @@ -61,6 +62,61 @@ func TestWalletSvrCmds(t *testing.T) { Account: btcjson.String("test"), }, }, + { + name: "createwallet", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("createwallet", "mywallet", true, true, "secret", true) + }, + staticCmd: func() interface{} { + return btcjson.NewCreateWalletCmd("mywallet", + btcjson.Bool(true), btcjson.Bool(true), + btcjson.String("secret"), btcjson.Bool(true)) + }, + marshalled: `{"jsonrpc":"1.0","method":"createwallet","params":["mywallet",true,true,"secret",true],"id":1}`, + unmarshalled: &btcjson.CreateWalletCmd{ + WalletName: "mywallet", + DisablePrivateKeys: btcjson.Bool(true), + Blank: btcjson.Bool(true), + Passphrase: btcjson.String("secret"), + AvoidReuse: btcjson.Bool(true), + }, + }, + { + name: "createwallet - optional1", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("createwallet", "mywallet") + }, + staticCmd: func() interface{} { + return btcjson.NewCreateWalletCmd("mywallet", + nil, nil, nil, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"createwallet","params":["mywallet"],"id":1}`, + unmarshalled: &btcjson.CreateWalletCmd{ + WalletName: "mywallet", + DisablePrivateKeys: btcjson.Bool(false), + Blank: btcjson.Bool(false), + Passphrase: btcjson.String(""), + AvoidReuse: btcjson.Bool(false), + }, + }, + { + name: "createwallet - optional2", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("createwallet", "mywallet", "null", "null", "secret") + }, + staticCmd: func() interface{} { + return btcjson.NewCreateWalletCmd("mywallet", + nil, nil, btcjson.String("secret"), nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"createwallet","params":["mywallet",null,null,"secret"],"id":1}`, + unmarshalled: &btcjson.CreateWalletCmd{ + WalletName: "mywallet", + DisablePrivateKeys: nil, + Blank: nil, + Passphrase: btcjson.String("secret"), + AvoidReuse: btcjson.Bool(false), + }, + }, { name: "addwitnessaddress", newCmd: func() (interface{}, error) { @@ -74,6 +130,49 @@ func TestWalletSvrCmds(t *testing.T) { Address: "1address", }, }, + { + name: "backupwallet", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("backupwallet", "backup.dat") + }, + staticCmd: func() interface{} { + return btcjson.NewBackupWalletCmd("backup.dat") + }, + marshalled: `{"jsonrpc":"1.0","method":"backupwallet","params":["backup.dat"],"id":1}`, + unmarshalled: &btcjson.BackupWalletCmd{Destination: "backup.dat"}, + }, + { + name: "loadwallet", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("loadwallet", "wallet.dat") + }, + staticCmd: func() interface{} { + return btcjson.NewLoadWalletCmd("wallet.dat") + }, + marshalled: `{"jsonrpc":"1.0","method":"loadwallet","params":["wallet.dat"],"id":1}`, + unmarshalled: &btcjson.LoadWalletCmd{WalletName: "wallet.dat"}, + }, + { + name: "unloadwallet", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("unloadwallet", "default") + }, + staticCmd: func() interface{} { + return btcjson.NewUnloadWalletCmd(btcjson.String("default")) + }, + marshalled: `{"jsonrpc":"1.0","method":"unloadwallet","params":["default"],"id":1}`, + unmarshalled: &btcjson.UnloadWalletCmd{WalletName: btcjson.String("default")}, + }, + {name: "unloadwallet - nil arg", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("unloadwallet") + }, + staticCmd: func() interface{} { + return btcjson.NewUnloadWalletCmd(nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"unloadwallet","params":[],"id":1}`, + unmarshalled: &btcjson.UnloadWalletCmd{WalletName: nil}, + }, { name: "createmultisig", newCmd: func() (interface{}, error) { @@ -208,6 +307,19 @@ func TestWalletSvrCmds(t *testing.T) { Account: "acct", }, }, + { + name: "getaddressinfo", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("getaddressinfo", "1234") + }, + staticCmd: func() interface{} { + return btcjson.NewGetAddressInfoCmd("1234") + }, + marshalled: `{"jsonrpc":"1.0","method":"getaddressinfo","params":["1234"],"id":1}`, + unmarshalled: &btcjson.GetAddressInfoCmd{ + Address: "1234", + }, + }, { name: "getbalance", newCmd: func() (interface{}, error) { @@ -707,6 +819,21 @@ func TestWalletSvrCmds(t *testing.T) { IncludeWatchOnly: btcjson.Bool(true), }, }, + { + name: "listsinceblock pad null", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("listsinceblock", "null", 1, false) + }, + staticCmd: func() interface{} { + return btcjson.NewListSinceBlockCmd(nil, btcjson.Int(1), btcjson.Bool(false)) + }, + marshalled: `{"jsonrpc":"1.0","method":"listsinceblock","params":[null,1,false],"id":1}`, + unmarshalled: &btcjson.ListSinceBlockCmd{ + BlockHash: nil, + TargetConfirmations: btcjson.Int(1), + IncludeWatchOnly: btcjson.Bool(false), + }, + }, { name: "listtransactions", newCmd: func() (interface{}, error) { @@ -1204,6 +1331,103 @@ func TestWalletSvrCmds(t *testing.T) { Flags: btcjson.String("ALL"), }, }, + { + name: "signrawtransactionwithwallet", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("signrawtransactionwithwallet", "001122") + }, + staticCmd: func() interface{} { + return btcjson.NewSignRawTransactionWithWalletCmd("001122", nil, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"signrawtransactionwithwallet","params":["001122"],"id":1}`, + unmarshalled: &btcjson.SignRawTransactionWithWalletCmd{ + RawTx: "001122", + Inputs: nil, + SigHashType: btcjson.String("ALL"), + }, + }, + { + name: "signrawtransactionwithwallet optional1", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("signrawtransactionwithwallet", "001122", `[{"txid":"123","vout":1,"scriptPubKey":"00","redeemScript":"01","witnessScript":"02","amount":1.5}]`) + }, + staticCmd: func() interface{} { + txInputs := []btcjson.RawTxWitnessInput{ + { + Txid: "123", + Vout: 1, + ScriptPubKey: "00", + RedeemScript: btcjson.String("01"), + WitnessScript: btcjson.String("02"), + Amount: btcjson.Float64(1.5), + }, + } + + return btcjson.NewSignRawTransactionWithWalletCmd("001122", &txInputs, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"signrawtransactionwithwallet","params":["001122",[{"txid":"123","vout":1,"scriptPubKey":"00","redeemScript":"01","witnessScript":"02","amount":1.5}]],"id":1}`, + unmarshalled: &btcjson.SignRawTransactionWithWalletCmd{ + RawTx: "001122", + Inputs: &[]btcjson.RawTxWitnessInput{ + { + Txid: "123", + Vout: 1, + ScriptPubKey: "00", + RedeemScript: btcjson.String("01"), + WitnessScript: btcjson.String("02"), + Amount: btcjson.Float64(1.5), + }, + }, + SigHashType: btcjson.String("ALL"), + }, + }, + { + name: "signrawtransactionwithwallet optional1 with blank fields in input", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("signrawtransactionwithwallet", "001122", `[{"txid":"123","vout":1,"scriptPubKey":"00","redeemScript":"01"}]`) + }, + staticCmd: func() interface{} { + txInputs := []btcjson.RawTxWitnessInput{ + { + Txid: "123", + Vout: 1, + ScriptPubKey: "00", + RedeemScript: btcjson.String("01"), + }, + } + + return btcjson.NewSignRawTransactionWithWalletCmd("001122", &txInputs, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"signrawtransactionwithwallet","params":["001122",[{"txid":"123","vout":1,"scriptPubKey":"00","redeemScript":"01"}]],"id":1}`, + unmarshalled: &btcjson.SignRawTransactionWithWalletCmd{ + RawTx: "001122", + Inputs: &[]btcjson.RawTxWitnessInput{ + { + Txid: "123", + Vout: 1, + ScriptPubKey: "00", + RedeemScript: btcjson.String("01"), + }, + }, + SigHashType: btcjson.String("ALL"), + }, + }, + { + name: "signrawtransactionwithwallet optional2", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd("signrawtransactionwithwallet", "001122", `[]`, "ALL") + }, + staticCmd: func() interface{} { + txInputs := []btcjson.RawTxWitnessInput{} + return btcjson.NewSignRawTransactionWithWalletCmd("001122", &txInputs, btcjson.String("ALL")) + }, + marshalled: `{"jsonrpc":"1.0","method":"signrawtransactionwithwallet","params":["001122",[],"ALL"],"id":1}`, + unmarshalled: &btcjson.SignRawTransactionWithWalletCmd{ + RawTx: "001122", + Inputs: &[]btcjson.RawTxWitnessInput{}, + SigHashType: btcjson.String("ALL"), + }, + }, { name: "walletlock", newCmd: func() (interface{}, error) { @@ -1243,13 +1467,340 @@ func TestWalletSvrCmds(t *testing.T) { NewPassphrase: "new", }, }, + { + name: "importmulti with descriptor + options", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp. + []btcjson.ImportMultiRequest{ + {Descriptor: btcjson.String("123"), Timestamp: btcjson.TimestampOrNow{Value: 0}}, + }, + `{"rescan": true}`, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + {Descriptor: btcjson.String("123"), Timestamp: btcjson.TimestampOrNow{Value: 0}}, + } + options := btcjson.ImportMultiOptions{Rescan: true} + return btcjson.NewImportMultiCmd(requests, &options) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"desc":"123","timestamp":0}],{"rescan":true}],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + }, + }, + Options: &btcjson.ImportMultiOptions{Rescan: true}, + }, + }, + { + name: "importmulti with descriptor + no options", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp. + []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + WatchOnly: btcjson.Bool(false), + Internal: btcjson.Bool(true), + Label: btcjson.String("aaa"), + KeyPool: btcjson.Bool(false), + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + WatchOnly: btcjson.Bool(false), + Internal: btcjson.Bool(true), + Label: btcjson.String("aaa"), + KeyPool: btcjson.Bool(false), + }, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"desc":"123","timestamp":0,"internal":true,"watchonly":false,"label":"aaa","keypool":false}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + WatchOnly: btcjson.Bool(false), + Internal: btcjson.Bool(true), + Label: btcjson.String("aaa"), + KeyPool: btcjson.Bool(false), + }, + }, + }, + }, + { + name: "importmulti with descriptor + string timestamp", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp. + []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: "now"}, + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + {Descriptor: btcjson.String("123"), Timestamp: btcjson.TimestampOrNow{Value: "now"}}, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"desc":"123","timestamp":"now"}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + {Descriptor: btcjson.String("123"), Timestamp: btcjson.TimestampOrNow{Value: "now"}}, + }, + }, + }, + { + name: "importmulti with scriptPubKey script", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp and scriptPubKey + []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: "script"}, + RedeemScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + PubKeys: &[]string{"aaa"}, + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: "script"}, + RedeemScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + PubKeys: &[]string{"aaa"}, + }, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"scriptPubKey":"script","timestamp":0,"redeemscript":"123","pubkeys":["aaa"]}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: "script"}, + RedeemScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + PubKeys: &[]string{"aaa"}, + }, + }, + }, + }, + { + name: "importmulti with scriptPubKey address", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp and scriptPubKey + []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: btcjson.ScriptPubKeyAddress{Address: "addr"}}, + WitnessScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Keys: &[]string{"aaa"}, + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: btcjson.ScriptPubKeyAddress{Address: "addr"}}, + WitnessScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Keys: &[]string{"aaa"}, + }, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"scriptPubKey":{"address":"addr"},"timestamp":0,"witnessscript":"123","keys":["aaa"]}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + ScriptPubKey: &btcjson.ScriptPubKey{Value: btcjson.ScriptPubKeyAddress{Address: "addr"}}, + WitnessScript: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Keys: &[]string{"aaa"}, + }, + }, + }, + }, + { + name: "importmulti with ranged (int) descriptor", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp. + []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: 7}, + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: 7}, + }, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"desc":"123","timestamp":0,"range":7}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: 7}, + }, + }, + }, + }, + { + name: "importmulti with ranged (slice) descriptor", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "importmulti", + // Cannot use a native string, due to special types like timestamp. + []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: []int{1, 7}}, + }, + }, + ) + }, + staticCmd: func() interface{} { + requests := []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: []int{1, 7}}, + }, + } + return btcjson.NewImportMultiCmd(requests, nil) + }, + marshalled: `{"jsonrpc":"1.0","method":"importmulti","params":[[{"desc":"123","timestamp":0,"range":[1,7]}]],"id":1}`, + unmarshalled: &btcjson.ImportMultiCmd{ + Requests: []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String("123"), + Timestamp: btcjson.TimestampOrNow{Value: 0}, + Range: &btcjson.DescriptorRange{Value: []int{1, 7}}, + }, + }, + }, + }, + { + name: "walletcreatefundedpsbt", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "walletcreatefundedpsbt", + []btcjson.PsbtInput{ + { + Txid: "1234", + Vout: 0, + Sequence: 0, + }, + }, + []btcjson.PsbtOutput{ + btcjson.NewPsbtOutput("1234", btcutil.Amount(1234)), + btcjson.NewPsbtDataOutput([]byte{1, 2, 3, 4}), + }, + btcjson.Uint32(1), + btcjson.WalletCreateFundedPsbtOpts{}, + btcjson.Bool(true), + ) + }, + staticCmd: func() interface{} { + return btcjson.NewWalletCreateFundedPsbtCmd( + []btcjson.PsbtInput{ + { + Txid: "1234", + Vout: 0, + Sequence: 0, + }, + }, + []btcjson.PsbtOutput{ + btcjson.NewPsbtOutput("1234", btcutil.Amount(1234)), + btcjson.NewPsbtDataOutput([]byte{1, 2, 3, 4}), + }, + btcjson.Uint32(1), + &btcjson.WalletCreateFundedPsbtOpts{}, + btcjson.Bool(true), + ) + }, + marshalled: `{"jsonrpc":"1.0","method":"walletcreatefundedpsbt","params":[[{"txid":"1234","vout":0,"sequence":0}],[{"1234":0.00001234},{"data":"01020304"}],1,{},true],"id":1}`, + unmarshalled: &btcjson.WalletCreateFundedPsbtCmd{ + Inputs: []btcjson.PsbtInput{ + { + Txid: "1234", + Vout: 0, + Sequence: 0, + }, + }, + Outputs: []btcjson.PsbtOutput{ + btcjson.NewPsbtOutput("1234", btcutil.Amount(1234)), + btcjson.NewPsbtDataOutput([]byte{1, 2, 3, 4}), + }, + Locktime: btcjson.Uint32(1), + Options: &btcjson.WalletCreateFundedPsbtOpts{}, + Bip32Derivs: btcjson.Bool(true), + }, + }, + { + name: "walletprocesspsbt", + newCmd: func() (interface{}, error) { + return btcjson.NewCmd( + "walletprocesspsbt", "1234", btcjson.Bool(true), btcjson.String("ALL"), btcjson.Bool(true)) + }, + staticCmd: func() interface{} { + return btcjson.NewWalletProcessPsbtCmd( + "1234", btcjson.Bool(true), btcjson.String("ALL"), btcjson.Bool(true)) + }, + marshalled: `{"jsonrpc":"1.0","method":"walletprocesspsbt","params":["1234",true,"ALL",true],"id":1}`, + unmarshalled: &btcjson.WalletProcessPsbtCmd{ + Psbt: "1234", + Sign: btcjson.Bool(true), + SighashType: btcjson.String("ALL"), + Bip32Derivs: btcjson.Bool(true), + }, + }, } t.Logf("Running %d tests", len(tests)) for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -1273,7 +1824,7 @@ func TestWalletSvrCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/walletsvrresults.go b/btcjson/walletsvrresults.go index 6e69ed9066..78a6e647f5 100644 --- a/btcjson/walletsvrresults.go +++ b/btcjson/walletsvrresults.go @@ -1,9 +1,132 @@ -// Copyright (c) 2014 The btcsuite developers +// Copyright (c) 2014-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. package btcjson +import ( + "encoding/json" + "fmt" + + "github.com/btcsuite/btcd/txscript" +) + +// CreateWalletResult models the result of the createwallet command. +type CreateWalletResult struct { + Name string `json:"name"` + Warning string `json:"warning"` +} + +// embeddedAddressInfo includes all getaddressinfo output fields, excluding +// metadata and relation to the wallet. +// +// It represents the non-metadata/non-wallet fields for GetAddressInfo, as well +// as the precise fields for an embedded P2SH or P2WSH address. +type embeddedAddressInfo struct { + Address string `json:"address"` + ScriptPubKey string `json:"scriptPubKey"` + Solvable bool `json:"solvable"` + Descriptor *string `json:"desc,omitempty"` + IsScript bool `json:"isscript"` + IsChange bool `json:"ischange"` + IsWitness bool `json:"iswitness"` + WitnessVersion int `json:"witness_version,omitempty"` + WitnessProgram *string `json:"witness_program,omitempty"` + ScriptType *txscript.ScriptClass `json:"script,omitempty"` + Hex *string `json:"hex,omitempty"` + PubKeys *[]string `json:"pubkeys,omitempty"` + SignaturesRequired *int `json:"sigsrequired,omitempty"` + PubKey *string `json:"pubkey,omitempty"` + IsCompressed *bool `json:"iscompressed,omitempty"` + HDMasterFingerprint *string `json:"hdmasterfingerprint,omitempty"` + Labels []string `json:"labels"` +} + +// GetAddressInfoResult models the result of the getaddressinfo command. It +// contains information about a bitcoin address. +// +// Reference: https://bitcoincore.org/en/doc/0.20.0/rpc/wallet/getaddressinfo +// +// The GetAddressInfoResult has three segments: +// 1. General information about the address. +// 2. Metadata (Timestamp, HDKeyPath, HDSeedID) and wallet fields +// (IsMine, IsWatchOnly). +// 3. Information about the embedded address in case of P2SH or P2WSH. +// Same structure as (1). +type GetAddressInfoResult struct { + embeddedAddressInfo + IsMine bool `json:"ismine"` + IsWatchOnly bool `json:"iswatchonly"` + Timestamp *int `json:"timestamp,omitempty"` + HDKeyPath *string `json:"hdkeypath,omitempty"` + HDSeedID *string `json:"hdseedid,omitempty"` + Embedded *embeddedAddressInfo `json:"embedded,omitempty"` +} + +// UnmarshalJSON provides a custom unmarshaller for GetAddressInfoResult. +// It is adapted to avoid creating a duplicate raw struct for unmarshalling +// the JSON bytes into. +// +// Reference: http://choly.ca/post/go-json-marshalling +func (e *GetAddressInfoResult) UnmarshalJSON(data []byte) error { + // Step 1: Create type aliases of the original struct, including the + // embedded one. + type Alias GetAddressInfoResult + type EmbeddedAlias embeddedAddressInfo + + // Step 2: Create an anonymous struct with raw replacements for the special + // fields. + aux := &struct { + ScriptType *string `json:"script,omitempty"` + Embedded *struct { + ScriptType *string `json:"script,omitempty"` + *EmbeddedAlias + } `json:"embedded,omitempty"` + *Alias + }{ + Alias: (*Alias)(e), + } + + // Step 3: Unmarshal the data into the anonymous struct. + if err := json.Unmarshal(data, &aux); err != nil { + return err + } + + // Step 4: Convert the raw fields to the desired types + var ( + sc *txscript.ScriptClass + err error + ) + + if aux.ScriptType != nil { + sc, err = txscript.NewScriptClass(*aux.ScriptType) + if err != nil { + return err + } + } + + e.ScriptType = sc + + if aux.Embedded != nil { + var ( + embeddedSc *txscript.ScriptClass + err error + ) + + if aux.Embedded.ScriptType != nil { + embeddedSc, err = txscript.NewScriptClass(*aux.Embedded.ScriptType) + if err != nil { + return err + } + } + + e.Embedded = (*embeddedAddressInfo)(aux.Embedded.EmbeddedAlias) + e.Embedded.ScriptType = embeddedSc + } + + return nil +} + // GetTransactionDetailsResult models the details data from the gettransaction command. // // This models the "short" version of the ListTransactionsResult type, which @@ -35,6 +158,58 @@ type GetTransactionResult struct { Hex string `json:"hex"` } +type ScanningOrFalse struct { + Value interface{} +} + +type ScanProgress struct { + Duration int `json:"duration"` + Progress float64 `json:"progress"` +} + +// MarshalJSON implements the json.Marshaler interface +func (h ScanningOrFalse) MarshalJSON() ([]byte, error) { + return json.Marshal(h.Value) +} + +// UnmarshalJSON implements the json.Unmarshaler interface +func (h *ScanningOrFalse) UnmarshalJSON(data []byte) error { + var unmarshalled interface{} + if err := json.Unmarshal(data, &unmarshalled); err != nil { + return err + } + + switch v := unmarshalled.(type) { + case bool: + h.Value = v + case map[string]interface{}: + h.Value = ScanProgress{ + Duration: int(v["duration"].(float64)), + Progress: v["progress"].(float64), + } + default: + return fmt.Errorf("invalid scanning value: %v", unmarshalled) + } + + return nil +} + +// GetWalletInfoResult models the result of the getwalletinfo command. +type GetWalletInfoResult struct { + WalletName string `json:"walletname"` + WalletVersion int `json:"walletversion"` + TransactionCount int `json:"txcount"` + KeyPoolOldest int `json:"keypoololdest"` + KeyPoolSize int `json:"keypoolsize"` + KeyPoolSizeHDInternal *int `json:"keypoolsize_hd_internal,omitempty"` + UnlockedUntil *int `json:"unlocked_until,omitempty"` + PayTransactionFee float64 `json:"paytxfee"` + HDSeedID *string `json:"hdseedid,omitempty"` + PrivateKeysEnabled bool `json:"private_keys_enabled"` + AvoidReuse bool `json:"avoid_reuse"` + Scanning ScanningOrFalse `json:"scanning"` +} + // InfoWalletResult models the data returned by the wallet server getinfo // command. type InfoWalletResult struct { @@ -64,6 +239,7 @@ type ListTransactionsResult struct { Amount float64 `json:"amount"` BIP125Replaceable string `json:"bip125-replaceable,omitempty"` BlockHash string `json:"blockhash,omitempty"` + BlockHeight *int32 `json:"blockheight,omitempty"` BlockIndex *int64 `json:"blockindex,omitempty"` BlockTime int64 `json:"blocktime,omitempty"` Category string `json:"category"` @@ -71,6 +247,7 @@ type ListTransactionsResult struct { Fee *float64 `json:"fee,omitempty"` Generated bool `json:"generated,omitempty"` InvolvesWatchOnly bool `json:"involveswatchonly,omitempty"` + Label *string `json:"label,omitempty"` Time int64 `json:"time"` TimeReceived int64 `json:"timereceived"` Trusted bool `json:"trusted"` @@ -137,6 +314,14 @@ type SignRawTransactionResult struct { Errors []SignRawTransactionError `json:"errors,omitempty"` } +// SignRawTransactionWithWalletResult models the data from the +// signrawtransactionwithwallet command. +type SignRawTransactionWithWalletResult struct { + Hex string `json:"hex"` + Complete bool `json:"complete"` + Errors []SignRawTransactionError `json:"errors,omitempty"` +} + // ValidateAddressWalletResult models the data returned by the wallet server // validateaddress command. type ValidateAddressWalletResult struct { @@ -173,3 +358,29 @@ type GetBalancesResult struct { Mine BalanceDetailsResult `json:"mine"` WatchOnly *BalanceDetailsResult `json:"watchonly"` } + +// ImportMultiResults is a slice that models the result of the importmulti command. +// +// Each item in the slice contains the execution result corresponding to the input +// requests of type btcjson.ImportMultiRequest, passed to the ImportMulti[Async] +// function. +type ImportMultiResults []struct { + Success bool `json:"success"` + Error *RPCError `json:"error,omitempty"` + Warnings *[]string `json:"warnings,omitempty"` +} + +// WalletCreateFundedPsbtResult models the data returned from the +// walletcreatefundedpsbtresult command. +type WalletCreateFundedPsbtResult struct { + Psbt string `json:"psbt"` + Fee float64 `json:"fee"` + ChangePos int64 `json:"changepos"` +} + +// WalletProcessPsbtResult models the data returned from the +// walletprocesspsbtresult command. +type WalletProcessPsbtResult struct { + Psbt string `json:"psbt"` + Complete bool `json:"complete"` +} diff --git a/btcjson/walletsvrresults_test.go b/btcjson/walletsvrresults_test.go new file mode 100644 index 0000000000..fd44b066b8 --- /dev/null +++ b/btcjson/walletsvrresults_test.go @@ -0,0 +1,127 @@ +// Copyright (c) 2020 The btcsuite developers +// Use of this source code is governed by an ISC +// license that can be found in the LICENSE file. + +package btcjson + +import ( + "encoding/json" + "errors" + "reflect" + "testing" + + "github.com/btcsuite/btcd/txscript" + "github.com/davecgh/go-spew/spew" +) + +// TestGetAddressInfoResult ensures that custom unmarshalling of +// GetAddressInfoResult works as intended. +func TestGetAddressInfoResult(t *testing.T) { + t.Parallel() + + // arbitrary script class to use in tests + nonStandard, _ := txscript.NewScriptClass("nonstandard") + + tests := []struct { + name string + result string + want GetAddressInfoResult + wantErr error + }{ + { + name: "GetAddressInfoResult - no ScriptType", + result: `{}`, + want: GetAddressInfoResult{}, + }, + { + name: "GetAddressInfoResult - ScriptType", + result: `{"script":"nonstandard","address":"1abc"}`, + want: GetAddressInfoResult{ + embeddedAddressInfo: embeddedAddressInfo{ + Address: "1abc", + ScriptType: nonStandard, + }, + }, + }, + { + name: "GetAddressInfoResult - embedded ScriptType", + result: `{"embedded": {"script":"nonstandard","address":"121313"}}`, + want: GetAddressInfoResult{ + Embedded: &embeddedAddressInfo{ + Address: "121313", + ScriptType: nonStandard, + }, + }, + }, + { + name: "GetAddressInfoResult - invalid ScriptType", + result: `{"embedded": {"script":"foo","address":"121313"}}`, + wantErr: txscript.ErrUnsupportedScriptType, + }, + } + + t.Logf("Running %d tests", len(tests)) + for i, test := range tests { + var out GetAddressInfoResult + err := json.Unmarshal([]byte(test.result), &out) + if err != nil && !errors.Is(err, test.wantErr) { + t.Errorf("Test #%d (%s) unexpected error: %v, want: %v", i, + test.name, err, test.wantErr) + continue + } + + if !reflect.DeepEqual(out, test.want) { + t.Errorf("Test #%d (%s) unexpected unmarshalled data - "+ + "got %v, want %v", i, test.name, spew.Sdump(out), + spew.Sdump(test.want)) + continue + } + } +} + +// TestGetWalletInfoResult ensures that custom unmarshalling of +// GetWalletInfoResult works as intended. +func TestGetWalletInfoResult(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + result string + want GetWalletInfoResult + }{ + { + name: "GetWalletInfoResult - not scanning", + result: `{"scanning":false}`, + want: GetWalletInfoResult{ + Scanning: ScanningOrFalse{Value: false}, + }, + }, + { + name: "GetWalletInfoResult - scanning", + result: `{"scanning":{"duration":10,"progress":1.0}}`, + want: GetWalletInfoResult{ + Scanning: ScanningOrFalse{ + Value: ScanProgress{Duration: 10, Progress: 1.0}, + }, + }, + }, + } + + t.Logf("Running %d tests", len(tests)) + for i, test := range tests { + var out GetWalletInfoResult + err := json.Unmarshal([]byte(test.result), &out) + if err != nil { + t.Errorf("Test #%d (%s) unexpected error: %v", i, + test.name, err) + continue + } + + if !reflect.DeepEqual(out, test.want) { + t.Errorf("Test #%d (%s) unexpected unmarshalled data - "+ + "got %v, want %v", i, test.name, spew.Sdump(out), + spew.Sdump(test.want)) + continue + } + } +} diff --git a/btcjson/walletsvrwscmds_test.go b/btcjson/walletsvrwscmds_test.go index 17144b6ea7..110a893b23 100644 --- a/btcjson/walletsvrwscmds_test.go +++ b/btcjson/walletsvrwscmds_test.go @@ -195,7 +195,7 @@ func TestWalletSvrWsCmds(t *testing.T) { for i, test := range tests { // Marshal the command as created by the new static command // creation function. - marshalled, err := btcjson.MarshalCmd(testID, test.staticCmd()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, testID, test.staticCmd()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -219,7 +219,7 @@ func TestWalletSvrWsCmds(t *testing.T) { // Marshal the command as created by the generic new command // creation function. - marshalled, err = btcjson.MarshalCmd(testID, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, testID, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/btcjson/walletsvrwsntfns_test.go b/btcjson/walletsvrwsntfns_test.go index 7662b3c2a1..111916627e 100644 --- a/btcjson/walletsvrwsntfns_test.go +++ b/btcjson/walletsvrwsntfns_test.go @@ -122,7 +122,7 @@ func TestWalletSvrWsNtfns(t *testing.T) { for i, test := range tests { // Marshal the notification as created by the new static // creation function. The ID is nil for notifications. - marshalled, err := btcjson.MarshalCmd(nil, test.staticNtfn()) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, test.staticNtfn()) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) @@ -147,7 +147,7 @@ func TestWalletSvrWsNtfns(t *testing.T) { // Marshal the notification as created by the generic new // notification creation function. The ID is nil for // notifications. - marshalled, err = btcjson.MarshalCmd(nil, cmd) + marshalled, err = btcjson.MarshalCmd(btcjson.RpcVersion1, nil, cmd) if err != nil { t.Errorf("MarshalCmd #%d (%s) unexpected error: %v", i, test.name, err) diff --git a/chaincfg/README.md b/chaincfg/README.md index 880446fdc7..72fac2e7dc 100644 --- a/chaincfg/README.md +++ b/chaincfg/README.md @@ -1,9 +1,9 @@ chaincfg ======== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/chaincfg) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/chaincfg) Package chaincfg defines chain configuration parameters for the three standard Bitcoin networks and provides the ability for callers to define their own custom diff --git a/chaincfg/chainhash/README.md b/chaincfg/chainhash/README.md index fc49d9cd93..b7ddf19ef7 100644 --- a/chaincfg/chainhash/README.md +++ b/chaincfg/chainhash/README.md @@ -1,9 +1,9 @@ chainhash ========= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/chaincfg/chainhash) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/chaincfg/chainhash) ======= chainhash provides a generic hash type and associated functions that allows the diff --git a/chaincfg/genesis.go b/chaincfg/genesis.go index ee47b84ce4..73d286102b 100644 --- a/chaincfg/genesis.go +++ b/chaincfg/genesis.go @@ -170,3 +170,31 @@ var simNetGenesisBlock = wire.MsgBlock{ }, Transactions: []*wire.MsgTx{&genesisCoinbaseTx}, } + +// sigNetGenesisHash is the hash of the first block in the block chain for the +// signet test network. +var sigNetGenesisHash = chainhash.Hash{ + 0xf6, 0x1e, 0xee, 0x3b, 0x63, 0xa3, 0x80, 0xa4, + 0x77, 0xa0, 0x63, 0xaf, 0x32, 0xb2, 0xbb, 0xc9, + 0x7c, 0x9f, 0xf9, 0xf0, 0x1f, 0x2c, 0x42, 0x25, + 0xe9, 0x73, 0x98, 0x81, 0x08, 0x00, 0x00, 0x00, +} + +// sigNetGenesisMerkleRoot is the hash of the first transaction in the genesis +// block for the signet test network. It is the same as the merkle root for +// the main network. +var sigNetGenesisMerkleRoot = genesisMerkleRoot + +// sigNetGenesisBlock defines the genesis block of the block chain which serves +// as the public transaction ledger for the signet test network. +var sigNetGenesisBlock = wire.MsgBlock{ + Header: wire.BlockHeader{ + Version: 1, + PrevBlock: chainhash.Hash{}, // 0000000000000000000000000000000000000000000000000000000000000000 + MerkleRoot: sigNetGenesisMerkleRoot, // 4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b + Timestamp: time.Unix(1598918400, 0), // 2020-09-01 00:00:00 +0000 UTC + Bits: 0x1e0377ae, // 503543726 [00000377ae000000000000000000000000000000000000000000000000000000] + Nonce: 52613770, + }, + Transactions: []*wire.MsgTx{&genesisCoinbaseTx}, +} diff --git a/chaincfg/genesis_test.go b/chaincfg/genesis_test.go index d04a72f758..1daf847916 100644 --- a/chaincfg/genesis_test.go +++ b/chaincfg/genesis_test.go @@ -118,6 +118,33 @@ func TestSimNetGenesisBlock(t *testing.T) { } } +// TestSigNetGenesisBlock tests the genesis block of the signet test network for +// validity by checking the encoded bytes and hashes. +func TestSigNetGenesisBlock(t *testing.T) { + // Encode the genesis block to raw bytes. + var buf bytes.Buffer + err := SigNetParams.GenesisBlock.Serialize(&buf) + if err != nil { + t.Fatalf("TestSigNetGenesisBlock: %v", err) + } + + // Ensure the encoded block matches the expected bytes. + if !bytes.Equal(buf.Bytes(), sigNetGenesisBlockBytes) { + t.Fatalf("TestSigNetGenesisBlock: Genesis block does not "+ + "appear valid - got %v, want %v", + spew.Sdump(buf.Bytes()), + spew.Sdump(sigNetGenesisBlockBytes)) + } + + // Check hash of the block against expected hash. + hash := SigNetParams.GenesisBlock.BlockHash() + if !SigNetParams.GenesisHash.IsEqual(&hash) { + t.Fatalf("TestSigNetGenesisBlock: Genesis block hash does "+ + "not appear valid - got %v, want %v", spew.Sdump(hash), + spew.Sdump(SigNetParams.GenesisHash)) + } +} + // genesisBlockBytes are the wire encoded bytes for the genesis block of the // main network as of protocol version 60002. var genesisBlockBytes = []byte{ @@ -281,3 +308,44 @@ var simNetGenesisBlockBytes = []byte{ 0x8a, 0x4c, 0x70, 0x2b, 0x6b, 0xf1, 0x1d, 0x5f, /* |.Lp+k.._|*/ 0xac, 0x00, 0x00, 0x00, 0x00, /* |.....| */ } + +// sigNetGenesisBlockBytes are the wire encoded bytes for the genesis block of +// the signet test network as of protocol version 70002. +var sigNetGenesisBlockBytes = []byte{ + 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |...@....| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x3b, 0xa3, 0xed, 0xfd, /* |........| */ + 0x7a, 0x7b, 0x12, 0xb2, 0x7a, 0xc7, 0x2c, 0x3e, /* |....;...| */ + 0x67, 0x76, 0x8f, 0x61, 0x7f, 0xc8, 0x1b, 0xc3, /* |z{..z.,>| */ + 0x88, 0x8a, 0x51, 0x32, 0x3a, 0x9f, 0xb8, 0xaa, /* |gv.a....| */ + 0x4b, 0x1e, 0x5e, 0x4a, 0x00, 0x8f, 0x4d, 0x5f, /* |..Q2:...| */ + 0xae, 0x77, 0x03, 0x1e, 0x8a, 0xd2, 0x22, 0x03, /* |K.^J..M_| */ + 0x01, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, /* |.w....".| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* |........| */ + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, /* |........| */ + 0xff, 0xff, 0x4d, 0x04, 0xff, 0xff, 0x00, 0x1d, /* |........| */ + 0x01, 0x04, 0x45, 0x54, 0x68, 0x65, 0x20, 0x54, /* |..M.....| */ + 0x69, 0x6d, 0x65, 0x73, 0x20, 0x30, 0x33, 0x2f, /* |..EThe T| */ + 0x4a, 0x61, 0x6e, 0x2f, 0x32, 0x30, 0x30, 0x39, /* |imes 03/| */ + 0x20, 0x43, 0x68, 0x61, 0x6e, 0x63, 0x65, 0x6c, /* |Jan/2009| */ + 0x6c, 0x6f, 0x72, 0x20, 0x6f, 0x6e, 0x20, 0x62, /* | Chancel| */ + 0x72, 0x69, 0x6e, 0x6b, 0x20, 0x6f, 0x66, 0x20, /* |lor on b| */ + 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x20, 0x62, /* |rink of| */ + 0x61, 0x69, 0x6c, 0x6f, 0x75, 0x74, 0x20, 0x66, /* |second b| */ + 0x6f, 0x72, 0x20, 0x62, 0x61, 0x6e, 0x6b, 0x73, /* |ailout f| */ + 0xff, 0xff, 0xff, 0xff, 0x01, 0x00, 0xf2, 0x05, /* |or banks| */ + 0x2a, 0x01, 0x00, 0x00, 0x00, 0x43, 0x41, 0x04, /* |........| */ + 0x67, 0x8a, 0xfd, 0xb0, 0xfe, 0x55, 0x48, 0x27, /* |*....CA.| */ + 0x19, 0x67, 0xf1, 0xa6, 0x71, 0x30, 0xb7, 0x10, /* |g....UH'| */ + 0x5c, 0xd6, 0xa8, 0x28, 0xe0, 0x39, 0x09, 0xa6, /* |.g..q0..| */ + 0x79, 0x62, 0xe0, 0xea, 0x1f, 0x61, 0xde, 0xb6, /* |\..(.9..| */ + 0x49, 0xf6, 0xbc, 0x3f, 0x4c, 0xef, 0x38, 0xc4, /* |yb...a..| */ + 0xf3, 0x55, 0x04, 0xe5, 0x1e, 0xc1, 0x12, 0xde, /* |I..?L.8.| */ + 0x5c, 0x38, 0x4d, 0xf7, 0xba, 0x0b, 0x8d, 0x57, /* |.U......| */ + 0x8a, 0x4c, 0x70, 0x2b, 0x6b, 0xf1, 0x1d, 0x5f, /* |\8M....W| */ + 0xac, 0x00, 0x00, 0x00, 0x00, /* |.....| */ +} diff --git a/chaincfg/params.go b/chaincfg/params.go index 54117b8713..a6d8d3e551 100644 --- a/chaincfg/params.go +++ b/chaincfg/params.go @@ -5,6 +5,8 @@ package chaincfg import ( + "encoding/binary" + "encoding/hex" "errors" "math" "math/big" @@ -38,6 +40,30 @@ var ( // simNetPowLimit is the highest proof of work value a Bitcoin block // can have for the simulation test network. It is the value 2^255 - 1. simNetPowLimit = new(big.Int).Sub(new(big.Int).Lsh(bigOne, 255), bigOne) + + // sigNetPowLimit is the highest proof of work value a bitcoin block can + // have for the signet test network. It is the value 0x0377ae << 216. + sigNetPowLimit = new(big.Int).Lsh(new(big.Int).SetInt64(0x0377ae), 216) + + // DefaultSignetChallenge is the byte representation of the signet + // challenge for the default (public, Taproot enabled) signet network. + // This is the binary equivalent of the bitcoin script + // 1 03ad5e0edad18cb1f0fc0d28a3d4f1f3e445640337489abb10404f2d1e086be430 + // 0359ef5021964fe22d6f8e05b2463c9540ce96883fe3b278760f048f5189f2e6c4 2 + // OP_CHECKMULTISIG + DefaultSignetChallenge, _ = hex.DecodeString( + "512103ad5e0edad18cb1f0fc0d28a3d4f1f3e445640337489abb10404f2d" + + "1e086be430210359ef5021964fe22d6f8e05b2463c9540ce9688" + + "3fe3b278760f048f5189f2e6c452ae", + ) + + // DefaultSignetDNSSeeds is the list of seed nodes for the default + // (public, Taproot enabled) signet network. + DefaultSignetDNSSeeds = []DNSSeed{ + {"178.128.221.177", false}, + {"2a01:7c8:d005:390::5", false}, + {"v7ajjeirttkbnt32wpy3c6w3emwnfr3fkla7hpxcfokr3ysd3kqtzmqd.onion:38333", false}, + } ) // Checkpoint identifies a known good point in the block chain. Using @@ -96,6 +122,11 @@ const ( // includes the deployment of BIPS 141, 142, 144, 145, 147 and 173. DeploymentSegwit + // DeploymentTaproot defines the rule change deployment ID for the + // Taproot (+Schnorr) soft-fork package. The taproot package includes + // the deployment of BIPS 340, 341 and 342. + DeploymentTaproot + // NOTE: DefinedDeployments must always come last since it is used to // determine how many defined deployments there currently are. @@ -578,6 +609,107 @@ var SimNetParams = Params{ HDCoinType: 115, // ASCII for s } +// SigNetParams defines the network parameters for the default public signet +// Bitcoin network. Not to be confused with the regression test network, this +// network is sometimes simply called "signet" or "taproot signet". +var SigNetParams = CustomSignetParams( + DefaultSignetChallenge, DefaultSignetDNSSeeds, +) + +// CustomSignetParams creates network parameters for a custom signet network +// from a challenge. The challenge is the binary compiled version of the block +// challenge script. +func CustomSignetParams(challenge []byte, dnsSeeds []DNSSeed) Params { + // The message start is defined as the first four bytes of the sha256d + // of the challenge script, as a single push (i.e. prefixed with the + // challenge script length). + challengeLength := byte(len(challenge)) + hashDouble := chainhash.DoubleHashB( + append([]byte{challengeLength}, challenge...), + ) + + // We use little endian encoding of the hash prefix to be in line with + // the other wire network identities. + net := binary.LittleEndian.Uint32(hashDouble[0:4]) + return Params{ + Name: "signet", + Net: wire.BitcoinNet(net), + DefaultPort: "38333", + DNSSeeds: dnsSeeds, + + // Chain parameters + GenesisBlock: &sigNetGenesisBlock, + GenesisHash: &sigNetGenesisHash, + PowLimit: sigNetPowLimit, + PowLimitBits: 0x1e0377ae, + BIP0034Height: 1, + BIP0065Height: 1, + BIP0066Height: 1, + CoinbaseMaturity: 100, + SubsidyReductionInterval: 210000, + TargetTimespan: time.Hour * 24 * 14, // 14 days + TargetTimePerBlock: time.Minute * 10, // 10 minutes + RetargetAdjustmentFactor: 4, // 25% less, 400% more + ReduceMinDifficulty: false, + MinDiffReductionTime: time.Minute * 20, // TargetTimePerBlock * 2 + GenerateSupported: false, + + // Checkpoints ordered from oldest to newest. + Checkpoints: nil, + + // Consensus rule change deployments. + // + // The miner confirmation window is defined as: + // target proof of work timespan / target proof of work spacing + RuleChangeActivationThreshold: 1916, // 95% of 2016 + MinerConfirmationWindow: 2016, + Deployments: [DefinedDeployments]ConsensusDeployment{ + DeploymentTestDummy: { + BitNumber: 28, + StartTime: 1199145601, // January 1, 2008 UTC + ExpireTime: 1230767999, // December 31, 2008 UTC + }, + DeploymentCSV: { + BitNumber: 29, + StartTime: 0, // Always available for vote + ExpireTime: math.MaxInt64, // Never expires + }, + DeploymentSegwit: { + BitNumber: 29, + StartTime: 0, // Always available for vote + ExpireTime: math.MaxInt64, // Never expires. + }, + DeploymentTaproot: { + BitNumber: 29, + StartTime: 0, // Always available for vote + ExpireTime: math.MaxInt64, // Never expires. + }, + }, + + // Mempool parameters + RelayNonStdTxs: false, + + // Human-readable part for Bech32 encoded segwit addresses, as defined in + // BIP 173. + Bech32HRPSegwit: "tb", // always tb for test net + + // Address encoding magics + PubKeyHashAddrID: 0x6f, // starts with m or n + ScriptHashAddrID: 0xc4, // starts with 2 + WitnessPubKeyHashAddrID: 0x03, // starts with QW + WitnessScriptHashAddrID: 0x28, // starts with T7n + PrivateKeyID: 0xef, // starts with 9 (uncompressed) or c (compressed) + + // BIP32 hierarchical deterministic extended key magics + HDPrivateKeyID: [4]byte{0x04, 0x35, 0x83, 0x94}, // starts with tprv + HDPublicKeyID: [4]byte{0x04, 0x35, 0x87, 0xcf}, // starts with tpub + + // BIP44 coin type used in the hierarchical deterministic path for + // address generation. + HDCoinType: 1, + } +} + var ( // ErrDuplicateNet describes an error where the parameters for a Bitcoin // network could not be set due to the network already being a standard @@ -588,6 +720,10 @@ var ( // is intended to identify the network for a hierarchical deterministic // private extended key is not registered. ErrUnknownHDKeyID = errors.New("unknown hd private extended key bytes") + + // ErrInvalidHDKeyID describes an error where the provided hierarchical + // deterministic version bytes, or hd key id, is malformed. + ErrInvalidHDKeyID = errors.New("invalid hd extended key version bytes") ) var ( @@ -619,7 +755,11 @@ func Register(params *Params) error { registeredNets[params.Net] = struct{}{} pubKeyHashAddrIDs[params.PubKeyHashAddrID] = struct{}{} scriptHashAddrIDs[params.ScriptHashAddrID] = struct{}{} - hdPrivToPubKeyIDs[params.HDPrivateKeyID] = params.HDPublicKeyID[:] + + err := RegisterHDKeyID(params.HDPublicKeyID[:], params.HDPrivateKeyID[:]) + if err != nil { + return err + } // A valid Bech32 encoded segwit address always has as prefix the // human-readable part for the given net followed by '1'. @@ -666,6 +806,29 @@ func IsBech32SegwitPrefix(prefix string) bool { return ok } +// RegisterHDKeyID registers a public and private hierarchical deterministic +// extended key ID pair. +// +// Non-standard HD version bytes, such as the ones documented in SLIP-0132, +// should be registered using this method for library packages to lookup key +// IDs (aka HD version bytes). When the provided key IDs are invalid, the +// ErrInvalidHDKeyID error will be returned. +// +// Reference: +// SLIP-0132 : Registered HD version bytes for BIP-0032 +// https://github.com/satoshilabs/slips/blob/master/slip-0132.md +func RegisterHDKeyID(hdPublicKeyID []byte, hdPrivateKeyID []byte) error { + if len(hdPublicKeyID) != 4 || len(hdPrivateKeyID) != 4 { + return ErrInvalidHDKeyID + } + + var keyID [4]byte + copy(keyID[:], hdPrivateKeyID) + hdPrivToPubKeyIDs[keyID] = hdPublicKeyID + + return nil +} + // HDPrivateKeyToPublicKeyID accepts a private hierarchical deterministic // extended key id and returns the associated public key id. When the provided // id is not registered, the ErrUnknownHDKeyID error will be returned. diff --git a/chaincfg/params_test.go b/chaincfg/params_test.go index 277a56bdd5..4166ce0a23 100644 --- a/chaincfg/params_test.go +++ b/chaincfg/params_test.go @@ -4,7 +4,12 @@ package chaincfg -import "testing" +import ( + "bytes" + "encoding/hex" + "math/big" + "testing" +) // TestInvalidHashStr ensures the newShaHashFromStr function panics when used to // with an invalid hash string. @@ -33,3 +38,98 @@ func TestMustRegisterPanic(t *testing.T) { // Intentionally try to register duplicate params to force a panic. mustRegister(&MainNetParams) } + +func TestRegisterHDKeyID(t *testing.T) { + t.Parallel() + + // Ref: https://github.com/satoshilabs/slips/blob/master/slip-0132.md + hdKeyIDZprv := []byte{0x02, 0xaa, 0x7a, 0x99} + hdKeyIDZpub := []byte{0x02, 0xaa, 0x7e, 0xd3} + + if err := RegisterHDKeyID(hdKeyIDZpub, hdKeyIDZprv); err != nil { + t.Fatalf("RegisterHDKeyID: expected no error, got %v", err) + } + + got, err := HDPrivateKeyToPublicKeyID(hdKeyIDZprv) + if err != nil { + t.Fatalf("HDPrivateKeyToPublicKeyID: expected no error, got %v", err) + } + + if !bytes.Equal(got, hdKeyIDZpub) { + t.Fatalf("HDPrivateKeyToPublicKeyID: expected result %v, got %v", + hdKeyIDZpub, got) + } +} + +func TestInvalidHDKeyID(t *testing.T) { + t.Parallel() + + prvValid := []byte{0x02, 0xaa, 0x7a, 0x99} + pubValid := []byte{0x02, 0xaa, 0x7e, 0xd3} + prvInvalid := []byte{0x00} + pubInvalid := []byte{0x00} + + if err := RegisterHDKeyID(pubInvalid, prvValid); err != ErrInvalidHDKeyID { + t.Fatalf("RegisterHDKeyID: want err ErrInvalidHDKeyID, got %v", err) + } + + if err := RegisterHDKeyID(pubValid, prvInvalid); err != ErrInvalidHDKeyID { + t.Fatalf("RegisterHDKeyID: want err ErrInvalidHDKeyID, got %v", err) + } + + if err := RegisterHDKeyID(pubInvalid, prvInvalid); err != ErrInvalidHDKeyID { + t.Fatalf("RegisterHDKeyID: want err ErrInvalidHDKeyID, got %v", err) + } + + // FIXME: The error type should be changed to ErrInvalidHDKeyID. + if _, err := HDPrivateKeyToPublicKeyID(prvInvalid); err != ErrUnknownHDKeyID { + t.Fatalf("HDPrivateKeyToPublicKeyID: want err ErrUnknownHDKeyID, got %v", err) + } +} + +func TestSigNetPowLimit(t *testing.T) { + sigNetPowLimitHex, _ := hex.DecodeString( + "00000377ae000000000000000000000000000000000000000000000000000000", + ) + powLimit := new(big.Int).SetBytes(sigNetPowLimitHex) + if sigNetPowLimit.Cmp(powLimit) != 0 { + t.Fatalf("Signet PoW limit bits (%s) not equal to big int (%s)", + sigNetPowLimit.Text(16), powLimit.Text(16)) + } + + if compactToBig(sigNetGenesisBlock.Header.Bits).Cmp(powLimit) != 0 { + t.Fatalf("Signet PoW limit header bits (%d) not equal to big "+ + "int (%s)", sigNetGenesisBlock.Header.Bits, + powLimit.Text(16)) + } +} + +// compactToBig is a copy of the blockchain.CompactToBig function. We copy it +// here so we don't run into a circular dependency just because of a test. +func compactToBig(compact uint32) *big.Int { + // Extract the mantissa, sign bit, and exponent. + mantissa := compact & 0x007fffff + isNegative := compact&0x00800000 != 0 + exponent := uint(compact >> 24) + + // Since the base for the exponent is 256, the exponent can be treated + // as the number of bytes to represent the full 256-bit number. So, + // treat the exponent as the number of bytes and shift the mantissa + // right or left accordingly. This is equivalent to: + // N = mantissa * 256^(exponent-3) + var bn *big.Int + if exponent <= 3 { + mantissa >>= 8 * (3 - exponent) + bn = big.NewInt(int64(mantissa)) + } else { + bn = big.NewInt(int64(mantissa)) + bn.Lsh(bn, 8*(exponent-3)) + } + + // Make it negative if the sign bit is set. + if isNegative { + bn = bn.Neg(bn) + } + + return bn +} diff --git a/cmd/addblock/addblock.go b/cmd/addblock/addblock.go index 15b61b6880..8b44f307e4 100644 --- a/cmd/addblock/addblock.go +++ b/cmd/addblock/addblock.go @@ -7,7 +7,6 @@ package main import ( "os" "path/filepath" - "runtime" "github.com/btcsuite/btcd/blockchain" "github.com/btcsuite/btcd/blockchain/indexers" @@ -119,8 +118,7 @@ func realMain() error { } func main() { - // Use all processor cores and up some limits. - runtime.GOMAXPROCS(runtime.NumCPU()) + // up some limits. if err := limits.SetLimits(); err != nil { os.Exit(1) } diff --git a/cmd/btcctl/btcctl.go b/cmd/btcctl/btcctl.go index 5c412f867f..771d5f7ed7 100644 --- a/cmd/btcctl/btcctl.go +++ b/cmd/btcctl/btcctl.go @@ -127,7 +127,7 @@ func main() { // Marshal the command into a JSON-RPC byte slice in preparation for // sending it to the RPC server. - marshalledJSON, err := btcjson.MarshalCmd(1, cmd) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, 1, cmd) if err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(1) diff --git a/cmd/btcctl/config.go b/cmd/btcctl/config.go index 939b6a8699..1cc2a260f3 100644 --- a/cmd/btcctl/config.go +++ b/cmd/btcctl/config.go @@ -107,6 +107,7 @@ type config struct { SimNet bool `long:"simnet" description:"Connect to the simulation test network"` TLSSkipVerify bool `long:"skipverify" description:"Do not verify tls certificates (not recommended!)"` TestNet3 bool `long:"testnet" description:"Connect to testnet"` + SigNet bool `long:"signet" description:"Connect to signet"` ShowVersion bool `short:"V" long:"version" description:"Display version information and exit"` Wallet bool `long:"wallet" description:"Connect to wallet"` } @@ -138,6 +139,12 @@ func normalizeAddress(addr string, chain *chaincfg.Params, useWallet bool) (stri } else { defaultPort = "18334" } + case &chaincfg.SigNetParams: + if useWallet { + defaultPort = "38332" + } else { + defaultPort = "38332" + } default: if useWallet { defaultPort = "8332" @@ -273,6 +280,10 @@ func loadConfig() (*config, []string, error) { numNets++ network = &chaincfg.RegressionNetParams } + if cfg.SigNet { + numNets++ + network = &chaincfg.SigNetParams + } if numNets > 1 { str := "%s: Multiple network params can't be used " + diff --git a/cmd/btcctl/version.go b/cmd/btcctl/version.go index f3a3e0b830..edb42dbe7d 100644 --- a/cmd/btcctl/version.go +++ b/cmd/btcctl/version.go @@ -17,7 +17,7 @@ const semanticAlphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqr // versioning 2.0.0 spec (http://semver.org/). const ( appMajor uint = 0 - appMinor uint = 21 + appMinor uint = 22 appPatch uint = 0 // appPreRelease MUST only contain characters from semanticAlphabet diff --git a/config.go b/config.go index c44f3a837f..7124fe9290 100644 --- a/config.go +++ b/config.go @@ -8,6 +8,7 @@ import ( "bufio" "crypto/rand" "encoding/base64" + "encoding/hex" "errors" "fmt" "io" @@ -159,6 +160,9 @@ type config struct { RPCUser string `short:"u" long:"rpcuser" description:"Username for RPC connections"` SigCacheMaxSize uint `long:"sigcachemaxsize" description:"The maximum number of entries in the signature verification cache"` SimNet bool `long:"simnet" description:"Use the simulation test network"` + SigNet bool `long:"signet" description:"Use the signet test network"` + SigNetChallenge string `long:"signetchallenge" description:"Connect to a custom signet network defined by this challenge instead of using the global default signet test network -- Can be specified multiple times"` + SigNetSeedNode []string `long:"signetseednode" description:"Specify a seed node for the signet network instead of using the global default signet network seed nodes"` TestNet3 bool `long:"testnet" description:"Use the test network"` TorIsolation bool `long:"torisolation" description:"Enable Tor stream isolation by randomizing user credentials for each connection."` TrickleInterval time.Duration `long:"trickleinterval" description:"Minimum time between attempts to send new inventory to a connected peer"` @@ -475,8 +479,8 @@ func loadConfig() (*config, []string, error) { // Load additional config from file. var configFileError error parser := newConfigParser(&cfg, &serviceOpts, flags.Default) - if !(preCfg.RegressionTest || preCfg.SimNet) || preCfg.ConfigFile != - defaultConfigFile { + if !(preCfg.RegressionTest || preCfg.SimNet || preCfg.SigNet) || + preCfg.ConfigFile != defaultConfigFile { if _, err := os.Stat(preCfg.ConfigFile); os.IsNotExist(err) { err := createDefaultConfigFile(preCfg.ConfigFile) @@ -550,9 +554,49 @@ func loadConfig() (*config, []string, error) { activeNetParams = &simNetParams cfg.DisableDNSSeed = true } + if cfg.SigNet { + numNets++ + activeNetParams = &sigNetParams + + // Let the user overwrite the default signet parameters. The + // challenge defines the actual signet network to join and the + // seed nodes are needed for network discovery. + sigNetChallenge := chaincfg.DefaultSignetChallenge + sigNetSeeds := chaincfg.DefaultSignetDNSSeeds + if cfg.SigNetChallenge != "" { + challenge, err := hex.DecodeString(cfg.SigNetChallenge) + if err != nil { + str := "%s: Invalid signet challenge, hex " + + "decode failed: %v" + err := fmt.Errorf(str, funcName, err) + fmt.Fprintln(os.Stderr, err) + fmt.Fprintln(os.Stderr, usageMessage) + return nil, nil, err + } + sigNetChallenge = challenge + } + + if len(cfg.SigNetSeedNode) > 0 { + sigNetSeeds = make( + []chaincfg.DNSSeed, len(cfg.SigNetSeedNode), + ) + for idx, seed := range cfg.SigNetSeedNode { + sigNetSeeds[idx] = chaincfg.DNSSeed{ + Host: seed, + HasFiltering: false, + } + } + } + + chainParams := chaincfg.CustomSignetParams( + sigNetChallenge, sigNetSeeds, + ) + activeNetParams.Params = &chainParams + } if numNets > 1 { - str := "%s: The testnet, regtest, segnet, and simnet params " + - "can't be used together -- choose one of the four" + str := "%s: The testnet, regtest, segnet, signet and simnet " + + "params can't be used together -- choose one of the " + + "five" err := fmt.Errorf(str, funcName) fmt.Fprintln(os.Stderr, err) fmt.Fprintln(os.Stderr, usageMessage) diff --git a/connmgr/README.md b/connmgr/README.md index 6f3968cecb..b1aa3cc7d0 100644 --- a/connmgr/README.md +++ b/connmgr/README.md @@ -1,9 +1,9 @@ connmgr ======= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/connmgr) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/connmgr) Package connmgr implements a generic Bitcoin network connection manager. diff --git a/database/README.md b/database/README.md index c51c9cc66b..21563d1af8 100644 --- a/database/README.md +++ b/database/README.md @@ -1,9 +1,9 @@ database ======== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/database) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/database) Package database provides a block and metadata storage database. @@ -42,11 +42,11 @@ $ go get -u github.com/btcsuite/btcd/database ## Examples -* [Basic Usage Example](http://godoc.org/github.com/btcsuite/btcd/database#example-package--BasicUsage) +* [Basic Usage Example](https://pkg.go.dev/github.com/btcsuite/btcd/database#example-package--BasicUsage) Demonstrates creating a new database and using a managed read-write transaction to store and retrieve metadata. -* [Block Storage and Retrieval Example](http://godoc.org/github.com/btcsuite/btcd/database#example-package--BlockStorageAndRetrieval) +* [Block Storage and Retrieval Example](https://pkg.go.dev/github.com/btcsuite/btcd/database#example-package--BlockStorageAndRetrieval) Demonstrates creating a new database, using a managed read-write transaction to store a block, and then using a managed read-only transaction to fetch the block. diff --git a/database/cmd/dbtool/main.go b/database/cmd/dbtool/main.go index 9eccd5989b..73c59a6e7d 100644 --- a/database/cmd/dbtool/main.go +++ b/database/cmd/dbtool/main.go @@ -7,7 +7,6 @@ package main import ( "os" "path/filepath" - "runtime" "strings" "github.com/btcsuite/btcd/database" @@ -106,9 +105,6 @@ func realMain() error { } func main() { - // Use all processor cores. - runtime.GOMAXPROCS(runtime.NumCPU()) - // Work around defer not working after os.Exit() if err := realMain(); err != nil { os.Exit(1) diff --git a/database/ffldb/README.md b/database/ffldb/README.md index ebaa5a7e01..5b855faa55 100644 --- a/database/ffldb/README.md +++ b/database/ffldb/README.md @@ -1,9 +1,9 @@ ffldb ===== -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://godoc.org/github.com/btcsuite/btcd/database/ffldb?status.png)](http://godoc.org/github.com/btcsuite/btcd/database/ffldb) +[![GoDoc](https://pkg.go.dev/github.com/btcsuite/btcd/database/ffldb?status.png)](https://pkg.go.dev/github.com/btcsuite/btcd/database/ffldb) ======= Package ffldb implements a driver for the database package that uses leveldb for diff --git a/database/ffldb/blockio.go b/database/ffldb/blockio.go index 3d7782f93c..8fb27ab283 100644 --- a/database/ffldb/blockio.go +++ b/database/ffldb/blockio.go @@ -474,7 +474,7 @@ func (s *blockStore) writeBlock(rawBlock []byte) (blockLocation, error) { _, _ = hasher.Write(scratch[:]) // Serialized block. - if err := s.writeData(rawBlock[:], "block"); err != nil { + if err := s.writeData(rawBlock, "block"); err != nil { return blockLocation{}, err } _, _ = hasher.Write(rawBlock) diff --git a/database/ffldb/driver_test.go b/database/ffldb/driver_test.go index 8ba6691af3..f3db909d79 100644 --- a/database/ffldb/driver_test.go +++ b/database/ffldb/driver_test.go @@ -9,7 +9,6 @@ import ( "os" "path/filepath" "reflect" - "runtime" "testing" "github.com/btcsuite/btcd/chaincfg" @@ -278,7 +277,6 @@ func TestInterface(t *testing.T) { } // Run all of the interface tests against the database. - runtime.GOMAXPROCS(runtime.NumCPU()) // Change the maximum file size to a small value to force multiple flat // files with the test data set. diff --git a/database/internal/treap/README.md b/database/internal/treap/README.md index f5314b4a05..14c3159a50 100644 --- a/database/internal/treap/README.md +++ b/database/internal/treap/README.md @@ -1,9 +1,9 @@ treap ===== -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://godoc.org/github.com/btcsuite/btcd/database/internal/treap?status.png)](http://godoc.org/github.com/btcsuite/btcd/database/internal/treap) +[![GoDoc](https://pkg.go.dev/github.com/btcsuite/btcd/database/internal/treap?status.png)](https://pkg.go.dev/github.com/btcsuite/btcd/database/internal/treap) Package treap implements a treap data structure that is used to hold ordered key/value pairs using a combination of binary search tree and heap semantics. diff --git a/doc.go b/doc.go index 8b9b99773a..70d0d9e45c 100644 --- a/doc.go +++ b/doc.go @@ -72,7 +72,7 @@ Application Options: minute (default: 15) --listen= Add an interface/port to listen for connections (default all interfaces port: 8333, testnet: - 18333) + 18333, signet: 38333) --logdir= Directory to log output --maxorphantx= Max number of orphan transactions to keep in memory (default: 100) diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 0000000000..a485625d42 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1 @@ +/_build diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000000..d4bb2cbb9e --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index afc58a631e..0000000000 --- a/docs/README.md +++ /dev/null @@ -1,296 +0,0 @@ -### Table of Contents -1. [About](#About) -2. [Getting Started](#GettingStarted) - 1. [Installation](#Installation) - 1. [Windows](#WindowsInstallation) - 2. [Linux/BSD/MacOSX/POSIX](#PosixInstallation) - 1. [Gentoo Linux](#GentooInstallation) - 2. [Configuration](#Configuration) - 3. [Controlling and Querying btcd via btcctl](#BtcctlConfig) - 4. [Mining](#Mining) -3. [Help](#Help) - 1. [Startup](#Startup) - 1. [Using bootstrap.dat](#BootstrapDat) - 2. [Network Configuration](#NetworkConfig) - 3. [Wallet](#Wallet) -4. [Contact](#Contact) - 1. [IRC](#ContactIRC) - 2. [Mailing Lists](#MailingLists) -5. [Developer Resources](#DeveloperResources) - 1. [Code Contribution Guidelines](#ContributionGuidelines) - 2. [JSON-RPC Reference](#JSONRPCReference) - 3. [The btcsuite Bitcoin-related Go Packages](#GoPackages) - - - -### 1. About - -btcd is a full node bitcoin implementation written in [Go](http://golang.org), -licensed under the [copyfree](http://www.copyfree.org) ISC License. - -This project is currently under active development and is in a Beta state. It -is extremely stable and has been in production use since October 2013. - -It properly downloads, validates, and serves the block chain using the exact -rules (including consensus bugs) for block acceptance as Bitcoin Core. We have -taken great care to avoid btcd causing a fork to the block chain. It includes a -full block validation testing framework which contains all of the 'official' -block acceptance tests (and some additional ones) that is run on every pull -request to help ensure it properly follows consensus. Also, it passes all of -the JSON test data in the Bitcoin Core code. - -It also properly relays newly mined blocks, maintains a transaction pool, and -relays individual transactions that have not yet made it into a block. It -ensures all individual transactions admitted to the pool follow the rules -required by the block chain and also includes more strict checks which filter -transactions based on miner requirements ("standard" transactions). - -One key difference between btcd and Bitcoin Core is that btcd does *NOT* include -wallet functionality and this was a very intentional design decision. See the -blog entry [here](https://web.archive.org/web/20171125143919/https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon) -for more details. This means you can't actually make or receive payments -directly with btcd. That functionality is provided by the -[btcwallet](https://github.com/btcsuite/btcwallet) and -[Paymetheus](https://github.com/btcsuite/Paymetheus) (Windows-only) projects -which are both under active development. - - - -### 2. Getting Started - - - -**2.1 Installation** - -The first step is to install btcd. See one of the following sections for -details on how to install on the supported operating systems. - - - -**2.1.1 Windows Installation**
- -* Install the MSI available at: https://github.com/btcsuite/btcd/releases -* Launch btcd from the Start Menu - - - -**2.1.2 Linux/BSD/MacOSX/POSIX Installation** - - -- Install Go according to the installation instructions here: - http://golang.org/doc/install - -- Ensure Go was installed properly and is a supported version: - -```bash -$ go version -$ go env GOROOT GOPATH -``` - -NOTE: The `GOROOT` and `GOPATH` above must not be the same path. It is -recommended that `GOPATH` is set to a directory in your home directory such as -`~/goprojects` to avoid write permission issues. It is also recommended to add -`$GOPATH/bin` to your `PATH` at this point. - -- Run the following commands to obtain btcd, all dependencies, and install it: - -```bash -$ git clone https://github.com/btcsuite/btcd $GOPATH/src/github.com/btcsuite/btcd -$ cd $GOPATH/src/github.com/btcsuite/btcd -$ GO111MODULE=on go install -v . ./cmd/... -``` - -- btcd (and utilities) will now be installed in ```$GOPATH/bin```. If you did - not already add the bin directory to your system path during Go installation, - we recommend you do so now. - -**Updating** - -- Run the following commands to update btcd, all dependencies, and install it: - -```bash -$ cd $GOPATH/src/github.com/btcsuite/btcd -$ git pull && GO111MODULE=on go install -v . ./cmd/... -``` - - - -**2.1.2.1 Gentoo Linux Installation** - -* Install Layman and enable the Bitcoin overlay. - * https://gitlab.com/bitcoin/gentoo -* Copy or symlink `/var/lib/layman/bitcoin/Documentation/package.keywords/btcd-live` to `/etc/portage/package.keywords/` -* Install btcd: `$ emerge net-p2p/btcd` - - - -**2.2 Configuration** - -btcd has a number of [configuration](http://godoc.org/github.com/btcsuite/btcd) -options, which can be viewed by running: `$ btcd --help`. - - - -**2.3 Controlling and Querying btcd via btcctl** - -btcctl is a command line utility that can be used to both control and query btcd -via [RPC](http://www.wikipedia.org/wiki/Remote_procedure_call). btcd does -**not** enable its RPC server by default; You must configure at minimum both an -RPC username and password or both an RPC limited username and password: - -* btcd.conf configuration file -``` -[Application Options] -rpcuser=myuser -rpcpass=SomeDecentp4ssw0rd -rpclimituser=mylimituser -rpclimitpass=Limitedp4ssw0rd -``` -* btcctl.conf configuration file -``` -[Application Options] -rpcuser=myuser -rpcpass=SomeDecentp4ssw0rd -``` -OR -``` -[Application Options] -rpclimituser=mylimituser -rpclimitpass=Limitedp4ssw0rd -``` -For a list of available options, run: `$ btcctl --help` - - - -**2.4 Mining** - -btcd supports the `getblocktemplate` RPC. -The limited user cannot access this RPC. - - -**1. Add the payment addresses with the `miningaddr` option.** - -``` -[Application Options] -rpcuser=myuser -rpcpass=SomeDecentp4ssw0rd -miningaddr=12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX -miningaddr=1M83ju3EChKYyysmM2FXtLNftbacagd8FR -``` - -**2. Add btcd's RPC TLS certificate to system Certificate Authority list.** - -`cgminer` uses [curl](http://curl.haxx.se/) to fetch data from the RPC server. -Since curl validates the certificate by default, we must install the `btcd` RPC -certificate into the default system Certificate Authority list. - -**Ubuntu** - -1. Copy rpc.cert to /usr/share/ca-certificates: `# cp /home/user/.btcd/rpc.cert /usr/share/ca-certificates/btcd.crt` -2. Add btcd.crt to /etc/ca-certificates.conf: `# echo btcd.crt >> /etc/ca-certificates.conf` -3. Update the CA certificate list: `# update-ca-certificates` - -**3. Set your mining software url to use https.** - -`$ cgminer -o https://127.0.0.1:8334 -u rpcuser -p rpcpassword` - - - -### 3. Help - - - -**3.1 Startup** - -Typically btcd will run and start downloading the block chain with no extra -configuration necessary, however, there is an optional method to use a -`bootstrap.dat` file that may speed up the initial block chain download process. - - - -**3.1.1 bootstrap.dat** - -* [Using bootstrap.dat](https://github.com/btcsuite/btcd/tree/master/docs/using_bootstrap_dat.md) - - - -**3.1.2 Network Configuration** - -* [What Ports Are Used by Default?](https://github.com/btcsuite/btcd/tree/master/docs/default_ports.md) -* [How To Listen on Specific Interfaces](https://github.com/btcsuite/btcd/tree/master/docs/configure_peer_server_listen_interfaces.md) -* [How To Configure RPC Server to Listen on Specific Interfaces](https://github.com/btcsuite/btcd/tree/master/docs/configure_rpc_server_listen_interfaces.md) -* [Configuring btcd with Tor](https://github.com/btcsuite/btcd/tree/master/docs/configuring_tor.md) - - - -**3.1 Wallet** - -btcd was intentionally developed without an integrated wallet for security -reasons. Please see [btcwallet](https://github.com/btcsuite/btcwallet) for more -information. - - - - -### 4. Contact - - - -**4.1 IRC** - -* [irc.freenode.net](irc://irc.freenode.net), channel `#btcd` - - - -**4.2 Mailing Lists** - -* btcd: discussion - of btcd and its packages. -* btcd-commits: - readonly mail-out of source code changes. - - - -### 5. Developer Resources - - - -* [Code Contribution Guidelines](https://github.com/btcsuite/btcd/tree/master/docs/code_contribution_guidelines.md) - - - -* [JSON-RPC Reference](https://github.com/btcsuite/btcd/tree/master/docs/json_rpc_api.md) - * [RPC Examples](https://github.com/btcsuite/btcd/tree/master/docs/json_rpc_api.md#ExampleCode) - - - -* The btcsuite Bitcoin-related Go Packages: - * [btcrpcclient](https://github.com/btcsuite/btcd/tree/master/rpcclient) - Implements a - robust and easy to use Websocket-enabled Bitcoin JSON-RPC client - * [btcjson](https://github.com/btcsuite/btcd/tree/master/btcjson) - Provides an extensive API - for the underlying JSON-RPC command and return values - * [wire](https://github.com/btcsuite/btcd/tree/master/wire) - Implements the - Bitcoin wire protocol - * [peer](https://github.com/btcsuite/btcd/tree/master/peer) - - Provides a common base for creating and managing Bitcoin network peers. - * [blockchain](https://github.com/btcsuite/btcd/tree/master/blockchain) - - Implements Bitcoin block handling and chain selection rules - * [blockchain/fullblocktests](https://github.com/btcsuite/btcd/tree/master/blockchain/fullblocktests) - - Provides a set of block tests for testing the consensus validation rules - * [txscript](https://github.com/btcsuite/btcd/tree/master/txscript) - - Implements the Bitcoin transaction scripting language - * [btcec](https://github.com/btcsuite/btcd/tree/master/btcec) - Implements - support for the elliptic curve cryptographic functions needed for the - Bitcoin scripts - * [database](https://github.com/btcsuite/btcd/tree/master/database) - - Provides a database interface for the Bitcoin block chain - * [mempool](https://github.com/btcsuite/btcd/tree/master/mempool) - - Package mempool provides a policy-enforced pool of unmined bitcoin - transactions. - * [btcutil](https://github.com/btcsuite/btcutil) - Provides Bitcoin-specific - convenience functions and types - * [chainhash](https://github.com/btcsuite/btcd/tree/master/chaincfg/chainhash) - - Provides a generic hash type and associated functions that allows the - specific hash algorithm to be abstracted. - * [connmgr](https://github.com/btcsuite/btcd/tree/master/connmgr) - - Package connmgr implements a generic Bitcoin network connection manager. diff --git a/docs/README.md b/docs/README.md new file mode 120000 index 0000000000..dd0ea36c8e --- /dev/null +++ b/docs/README.md @@ -0,0 +1 @@ +index.md \ No newline at end of file diff --git a/docs/code_contribution_guidelines.md b/docs/code_contribution_guidelines.md index b643e911a3..c0a7eecc5f 100644 --- a/docs/code_contribution_guidelines.md +++ b/docs/code_contribution_guidelines.md @@ -1,23 +1,4 @@ -### Table of Contents -1. [Overview](#Overview)
-2. [Minimum Recommended Skillset](#MinSkillset)
-3. [Required Reading](#ReqReading)
-4. [Development Practices](#DevelopmentPractices)
-4.1. [Share Early, Share Often](#ShareEarly)
-4.2. [Testing](#Testing)
-4.3. [Code Documentation and Commenting](#CodeDocumentation)
-4.4. [Model Git Commit Messages](#ModelGitCommitMessages)
-5. [Code Approval Process](#CodeApproval)
-5.1 [Code Review](#CodeReview)
-5.2 [Rework Code (if needed)](#CodeRework)
-5.3 [Acceptance](#CodeAcceptance)
-6. [Contribution Standards](#Standards)
-6.1. [Contribution Checklist](#Checklist)
-6.2. [Licensing of Contributions](#Licensing)
- - - -### 1. Overview +# Code contribution guidelines Developing cryptocurrencies is an exciting endeavor that touches a wide variety of areas such as wire protocols, peer-to-peer networking, databases, @@ -38,9 +19,7 @@ is outlined on this page. We highly encourage code contributions, however it is imperative that you adhere to the guidelines established on this page. - - -### 2. Minimum Recommended Skillset +## Minimum Recommended Skillset The following list is a set of core competencies that we recommend you possess before you really start attempting to contribute code to the project. These are @@ -64,9 +43,7 @@ if you wish to contribute to the cryptography code, you should have a good understanding of the various aspects involved with cryptography such as the security and performance implications. - - -### 3. Required Reading +## Required Reading - [Effective Go](http://golang.org/doc/effective_go.html) - The entire btcd suite follows the guidelines in this document. For your code to be accepted, @@ -74,17 +51,13 @@ security and performance implications. - [Original Satoshi Whitepaper](http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CCkQFjAA&url=http%3A%2F%2Fbitcoin.org%2Fbitcoin.pdf&ei=os3VUuH8G4SlsASV74GoAg&usg=AFQjCNEipPLigou_1MfB7DQjXCNdlylrBg&sig2=FaHDuT5z36GMWDEnybDJLg&bvm=bv.59378465,d.b2I) - This is the white paper that started it all. Having a solid foundation to build on will make the code much more comprehensible. - - -### 4. Development Practices +## Development Practices Developers are expected to work in their own trees and submit pull requests when they feel their feature or bug fix is ready for integration into the master branch. - - -### 4.1 Share Early, Share Often +## Share Early, Share Often We firmly believe in the share early, share often approach. The basic premise of the approach is to announce your plans **before** you start work, and once @@ -105,9 +78,7 @@ This approach has several benefits: - The quicker your changes are merged to master, the less time you will need to spend rebasing and otherwise trying to keep up with the main code base - - -### 4.2 Testing +## Testing One of the major design goals of all core btcd packages is to aim for complete test coverage. This is financial software so bugs and regressions can cost @@ -126,34 +97,37 @@ checking coverage statistics straight forward. For more information about the test coverage tools, see the [golang cover blog post](http://blog.golang.org/cover). A quick summary of test practices follows: + - All new code should be accompanied by tests that ensure the code behaves correctly when given expected values, and, perhaps even more importantly, that it handles errors gracefully - When you fix a bug, it should be accompanied by tests which exercise the bug to both prove it has been resolved and to prevent future regressions - - -### 4.3 Code Documentation and Commenting +## Code Documentation and Commenting - At a minimum every function must be commented with its intended purpose and any assumptions that it makes - Function comments must always begin with the name of the function per [Effective Go](http://golang.org/doc/effective_go.html) - Function comments should be complete sentences since they allow a wide - variety of automated presentations such as [godoc.org](https://godoc.org) + variety of automated presentations such as [go.dev](https://go.dev) - The general rule of thumb is to look at it as if you were completely unfamiliar with the code and ask yourself, would this give me enough - information to understand what this function does and how I'd probably want - to use it? + information to understand what this function does and how I'd probably want + to use it? - Exported functions should also include detailed information the caller of the - function will likely need to know and/or understand:

+ function will likely need to know and/or understand: + **WRONG** + ```Go // convert a compact uint32 to big.Int func CompactToBig(compact uint32) *big.Int { ``` + **RIGHT** + ```Go // CompactToBig converts a compact representation of a whole number N to a // big integer. The representation is similar to IEEE754 floating point @@ -180,31 +154,35 @@ func CompactToBig(compact uint32) *big.Int { // sign bit, but it is implemented here to stay consistent with bitcoind. func CompactToBig(compact uint32) *big.Int { ``` + - Comments in the body of the code are highly encouraged, but they should explain the intention of the code as opposed to just calling out the - obvious

+ obvious + **WRONG** + ```Go // return err if amt is less than 5460 if amt < 5460 { - return err + return err } ``` + **RIGHT** + ```Go // Treat transactions with amounts less than the amount which is considered dust // as non-standard. if amt < 5460 { - return err + return err } ``` + **NOTE:** The above should really use a constant as opposed to a magic number, but it was left as a magic number to show how much of a difference a good comment can make. - - -### 4.4 Model Git Commit Messages +## Model Git Commit Messages This project prefers to keep a clean commit history with well-formed commit messages. This section illustrates a model commit message and provides a bit @@ -214,7 +192,7 @@ being provided here. Here’s a model Git commit message: -``` +```text Short (50 chars or less) summary of changes More detailed explanatory text, if necessary. Wrap it to about 72 @@ -255,22 +233,18 @@ a good thing. wrap our plain text emails such that there’s room for a few levels of nested reply indicators without overflow in an 80 column terminal. - - -### 5. Code Approval Process +## Code Approval Process This section describes the code approval process that is used for code contributions. This is how to get your changes into btcd. - - -### 5.1 Code Review +## Code Review All code which is submitted will need to be reviewed before inclusion into the master branch. This process is performed by the project maintainers and usually other committers who are interested in the area you are working in as well. -##### Code Review Timeframe +## Code Review Timeframe The timeframe for a code review will vary greatly depending on factors such as the number of other pull requests which need to be reviewed, the size and @@ -286,7 +260,7 @@ days, while large or far reaching changes may take weeks. This is a good reason to stick with the [Share Early, Share Often](#ShareOften) development practice outlined above. -##### What is the review looking for? +## What is the review looking for? The review is mainly ensuring the code follows the [Development Practices](#DevelopmentPractices) and [Code Contribution Standards](#Standards). However, there are a few other @@ -298,9 +272,7 @@ checks which are generally performed as follows: - The change is not something which is deemed inappropriate by community consensus - - -### 5.2 Rework Code (if needed) +## Rework Code (if needed) After the code review, the change will be accepted immediately if no issues are found. If there are any concerns or questions, you will be provided with @@ -311,9 +283,7 @@ make the necessary changes. This process will continue until the code is finally accepted. - - -### 5.3 Acceptance +## Acceptance Once your code is accepted, it will be integrated with the master branch. Typically it will be rebased and fast-forward merged to master as we prefer to @@ -323,13 +293,9 @@ the master branch and the pull request will be closed. Rejoice as you will now be listed as a [contributor](https://github.com/btcsuite/btcd/graphs/contributors)! - +## Contribution Standards -### 6. Contribution Standards - - - -### 6.1. Contribution Checklist +## Contribution Checklist - [  ] All changes are Go version 1.3 compliant - [  ] The code being submitted is commented according to the @@ -346,9 +312,7 @@ Rejoice as you will now be listed as a [contributor](https://github.com/btcsuite - [  ] Running [golint](https://github.com/golang/lint) does not report any **new** issues that did not already exist - - -### 6.2. Licensing of Contributions +## Licensing of Contributions All contributions must be licensed with the [ISC license](https://github.com/btcsuite/btcd/blob/master/LICENSE). This is diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000000..3db163052b --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,77 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) +import recommonmark +from recommonmark.transform import AutoStructify + +# -- Project information ----------------------------------------------------- + +project = 'btcd' +copyright = '2020, btcd' +author = 'btcsuite developers' + +# The full version, including alpha/beta/rc tags +release = 'beta' + +source_suffix = ['.md'] + +# The master toctree document. +master_doc = 'index' + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.napoleon', + 'sphinx.ext.mathjax', + 'sphinx_markdown_tables', + 'recommonmark', +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# app setup hook +def setup(app): + app.add_config_value('recommonmark_config', { + #'url_resolver': lambda url: github_doc_root + url, + 'auto_toc_tree_section': 'Contents', + 'enable_math': False, + 'enable_inline_math': False, + 'enable_eval_rst': True, + 'enable_auto_doc_ref': True, + }, True) + app.add_transform(AutoStructify) diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000000..c6f95b274c --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,190 @@ +# Configuration + +btcd has a number of [configuration](https://pkg.go.dev/github.com/btcsuite/btcd) +options, which can be viewed by running: `$ btcd --help`. + +## Peer server listen interface + +btcd allows you to bind to specific interfaces which enables you to setup +configurations with varying levels of complexity. The listen parameter can be +specified on the command line as shown below with the -- prefix or in the +configuration file without the -- prefix (as can all long command line options). +The configuration file takes one entry per line. + +**NOTE:** The listen flag can be specified multiple times to listen on multiple +interfaces as a couple of the examples below illustrate. + +Command Line Examples: + +|Flags|Comment| +|----------|------------| +|--listen=|all interfaces on default port which is changed by `--testnet` and `--regtest` (**default**)| +|--listen=0.0.0.0|all IPv4 interfaces on default port which is changed by `--testnet` and `--regtest`| +|--listen=::|all IPv6 interfaces on default port which is changed by `--testnet` and `--regtest`| +|--listen=:8333|all interfaces on port 8333| +|--listen=0.0.0.0:8333|all IPv4 interfaces on port 8333| +|--listen=[::]:8333|all IPv6 interfaces on port 8333| +|--listen=127.0.0.1:8333|only IPv4 localhost on port 8333| +|--listen=[::1]:8333|only IPv6 localhost on port 8333| +|--listen=:8336|all interfaces on non-standard port 8336| +|--listen=0.0.0.0:8336|all IPv4 interfaces on non-standard port 8336| +|--listen=[::]:8336|all IPv6 interfaces on non-standard port 8336| +|--listen=127.0.0.1:8337 --listen=[::1]:8333|IPv4 localhost on port 8337 and IPv6 localhost on port 8333| +|--listen=:8333 --listen=:8337|all interfaces on ports 8333 and 8337| + +The following config file would configure btcd to only listen on localhost for both IPv4 and IPv6: + +```text +[Application Options] + +listen=127.0.0.1:8333 +listen=[::1]:8333 +``` + +In addition, if you are starting btcd with TLS and want to make it +available via a hostname, then you will need to generate the TLS +certificates for that host. For example, + +``` +gencerts --host=myhostname.example.com --directory=/home/me/.btcd/ +``` + +## RPC server listen interface + +btcd allows you to bind the RPC server to specific interfaces which enables you +to setup configurations with varying levels of complexity. The `rpclisten` +parameter can be specified on the command line as shown below with the -- prefix +or in the configuration file without the -- prefix (as can all long command line +options). The configuration file takes one entry per line. + +A few things to note regarding the RPC server: + +* The RPC server will **not** be enabled unless the `rpcuser` and `rpcpass` + options are specified. +* When the `rpcuser` and `rpcpass` and/or `rpclimituser` and `rpclimitpass` + options are specified, the RPC server will only listen on localhost IPv4 and + IPv6 interfaces by default. You will need to override the RPC listen + interfaces to include external interfaces if you want to connect from a remote + machine. +* The RPC server has TLS enabled by default, even for localhost. You may use + the `--notls` option to disable it, but only when all listeners are on + localhost interfaces. +* The `--rpclisten` flag can be specified multiple times to listen on multiple + interfaces as a couple of the examples below illustrate. +* The RPC server is disabled by default when using the `--regtest` and + `--simnet` networks. You can override this by specifying listen interfaces. + +Command Line Examples: + +|Flags|Comment| +|----------|------------| +|--rpclisten=|all interfaces on default port which is changed by `--testnet`| +|--rpclisten=0.0.0.0|all IPv4 interfaces on default port which is changed by `--testnet`| +|--rpclisten=::|all IPv6 interfaces on default port which is changed by `--testnet`| +|--rpclisten=:8334|all interfaces on port 8334| +|--rpclisten=0.0.0.0:8334|all IPv4 interfaces on port 8334| +|--rpclisten=[::]:8334|all IPv6 interfaces on port 8334| +|--rpclisten=127.0.0.1:8334|only IPv4 localhost on port 8334| +|--rpclisten=[::1]:8334|only IPv6 localhost on port 8334| +|--rpclisten=:8336|all interfaces on non-standard port 8336| +|--rpclisten=0.0.0.0:8336|all IPv4 interfaces on non-standard port 8336| +|--rpclisten=[::]:8336|all IPv6 interfaces on non-standard port 8336| +|--rpclisten=127.0.0.1:8337 --listen=[::1]:8334|IPv4 localhost on port 8337 and IPv6 localhost on port 8334| +|--rpclisten=:8334 --listen=:8337|all interfaces on ports 8334 and 8337| + +The following config file would configure the btcd RPC server to listen to all interfaces on the default port, including external interfaces, for both IPv4 and IPv6: + +```text +[Application Options] + +rpclisten= +``` + +## Default ports + +While btcd is highly configurable when it comes to the network configuration, +the following is intended to be a quick reference for the default ports used so +port forwarding can be configured as required. + +btcd provides a `--upnp` flag which can be used to automatically map the bitcoin +peer-to-peer listening port if your router supports UPnP. If your router does +not support UPnP, or you don't wish to use it, please note that only the bitcoin +peer-to-peer port should be forwarded unless you specifically want to allow RPC +access to your btcd from external sources such as in more advanced network +configurations. + +|Name|Port| +|----|----| +|Default Bitcoin peer-to-peer port|TCP 8333| +|Default RPC port|TCP 8334| + +## Using bootstrap.dat + +### What is bootstrap.dat? + +It is a flat, binary file containing bitcoin blockchain data starting from the +genesis block and continuing through a relatively recent block height depending +on the last time it was updated. + +See [this](https://bitcointalk.org/index.php?topic=145386.0) thread on +bitcointalk for more details. + +**NOTE:** Using bootstrap.dat is entirely optional. Btcd will download the +block chain from other peers through the Bitcoin protocol with no extra +configuration needed. + +### What are the pros and cons of using bootstrap.dat? + +Pros: + +* Typically accelerates the initial process of bringing up a new node as it + downloads from public P2P nodes and generally is able to achieve faster + download speeds +* It is particularly beneficial when bringing up multiple nodes as you only need + to download the data once + +Cons: + +* Requires you to setup and configure a torrent client if you don't already have + one available +* Requires roughly twice as much disk space since you'll need the flat file as + well as the imported database + +### Where do I get bootstrap.dat? + +The bootstrap.dat file is made available via a torrent. See +[this](https://bitcointalk.org/index.php?topic=145386.0) thread on bitcointalk +for the torrent download details. + +### How do I know I can trust the bootstrap.dat I downloaded? + +You don't need to trust the file as the `addblock` utility verifies every block +using the same rules that are used when downloading the block chain normally +through the Bitcoin protocol. Additionally, the chain rules contain hard-coded +checkpoints for the known-good block chain at periodic intervals. This ensures +that not only is it a valid chain, but it is the same chain that everyone else +is using. + +### How do I use bootstrap.dat with btcd? + +btcd comes with a separate utility named `addblock` which can be used to import +`bootstrap.dat`. This approach is used since the import is a one-time operation +and we prefer to keep the daemon itself as lightweight as possible. + +1. Stop btcd if it is already running. This is required since addblock needs to + access the database used by btcd and it will be locked if btcd is using it. +2. Note the path to the downloaded bootstrap.dat file. +3. Run the addblock utility with the `-i` argument pointing to the location of + boostrap.dat: + +**Windows:** + +```bat +"%PROGRAMFILES%\Btcd Suite\Btcd\addblock" -i C:\Path\To\bootstrap.dat +``` + +**Linux/Unix/BSD/POSIX:** + +```bash +$GOPATH/bin/addblock -i /path/to/bootstrap.dat +``` diff --git a/docs/configure_peer_server_listen_interfaces.md b/docs/configure_peer_server_listen_interfaces.md deleted file mode 100644 index ac61137335..0000000000 --- a/docs/configure_peer_server_listen_interfaces.md +++ /dev/null @@ -1,43 +0,0 @@ -btcd allows you to bind to specific interfaces which enables you to setup -configurations with varying levels of complexity. The listen parameter can be -specified on the command line as shown below with the -- prefix or in the -configuration file without the -- prefix (as can all long command line options). -The configuration file takes one entry per line. - -**NOTE:** The listen flag can be specified multiple times to listen on multiple -interfaces as a couple of the examples below illustrate. - -Command Line Examples: - -|Flags|Comment| -|----------|------------| -|--listen=|all interfaces on default port which is changed by `--testnet` and `--regtest` (**default**)| -|--listen=0.0.0.0|all IPv4 interfaces on default port which is changed by `--testnet` and `--regtest`| -|--listen=::|all IPv6 interfaces on default port which is changed by `--testnet` and `--regtest`| -|--listen=:8333|all interfaces on port 8333| -|--listen=0.0.0.0:8333|all IPv4 interfaces on port 8333| -|--listen=[::]:8333|all IPv6 interfaces on port 8333| -|--listen=127.0.0.1:8333|only IPv4 localhost on port 8333| -|--listen=[::1]:8333|only IPv6 localhost on port 8333| -|--listen=:8336|all interfaces on non-standard port 8336| -|--listen=0.0.0.0:8336|all IPv4 interfaces on non-standard port 8336| -|--listen=[::]:8336|all IPv6 interfaces on non-standard port 8336| -|--listen=127.0.0.1:8337 --listen=[::1]:8333|IPv4 localhost on port 8337 and IPv6 localhost on port 8333| -|--listen=:8333 --listen=:8337|all interfaces on ports 8333 and 8337| - -The following config file would configure btcd to only listen on localhost for both IPv4 and IPv6: - -```text -[Application Options] - -listen=127.0.0.1:8333 -listen=[::1]:8333 -``` - -In addition, if you are starting btcd with TLS and want to make it -available via a hostname, then you will need to generate the TLS -certificates for that host. For example, - -``` -gencerts --host=myhostname.example.com --directory=/home/me/.btcd/ -``` diff --git a/docs/configure_rpc_server_listen_interfaces.md b/docs/configure_rpc_server_listen_interfaces.md deleted file mode 100644 index 3115d6a16f..0000000000 --- a/docs/configure_rpc_server_listen_interfaces.md +++ /dev/null @@ -1,47 +0,0 @@ -btcd allows you to bind the RPC server to specific interfaces which enables you -to setup configurations with varying levels of complexity. The `rpclisten` -parameter can be specified on the command line as shown below with the -- prefix -or in the configuration file without the -- prefix (as can all long command line -options). The configuration file takes one entry per line. - -A few things to note regarding the RPC server: -* The RPC server will **not** be enabled unless the `rpcuser` and `rpcpass` - options are specified. -* When the `rpcuser` and `rpcpass` and/or `rpclimituser` and `rpclimitpass` - options are specified, the RPC server will only listen on localhost IPv4 and - IPv6 interfaces by default. You will need to override the RPC listen - interfaces to include external interfaces if you want to connect from a remote - machine. -* The RPC server has TLS enabled by default, even for localhost. You may use - the `--notls` option to disable it, but only when all listeners are on - localhost interfaces. -* The `--rpclisten` flag can be specified multiple times to listen on multiple - interfaces as a couple of the examples below illustrate. -* The RPC server is disabled by default when using the `--regtest` and - `--simnet` networks. You can override this by specifying listen interfaces. - -Command Line Examples: - -|Flags|Comment| -|----------|------------| -|--rpclisten=|all interfaces on default port which is changed by `--testnet`| -|--rpclisten=0.0.0.0|all IPv4 interfaces on default port which is changed by `--testnet`| -|--rpclisten=::|all IPv6 interfaces on default port which is changed by `--testnet`| -|--rpclisten=:8334|all interfaces on port 8334| -|--rpclisten=0.0.0.0:8334|all IPv4 interfaces on port 8334| -|--rpclisten=[::]:8334|all IPv6 interfaces on port 8334| -|--rpclisten=127.0.0.1:8334|only IPv4 localhost on port 8334| -|--rpclisten=[::1]:8334|only IPv6 localhost on port 8334| -|--rpclisten=:8336|all interfaces on non-standard port 8336| -|--rpclisten=0.0.0.0:8336|all IPv4 interfaces on non-standard port 8336| -|--rpclisten=[::]:8336|all IPv6 interfaces on non-standard port 8336| -|--rpclisten=127.0.0.1:8337 --listen=[::1]:8334|IPv4 localhost on port 8337 and IPv6 localhost on port 8334| -|--rpclisten=:8334 --listen=:8337|all interfaces on ports 8334 and 8337| - -The following config file would configure the btcd RPC server to listen to all interfaces on the default port, including external interfaces, for both IPv4 and IPv6: - -```text -[Application Options] - -rpclisten= -``` diff --git a/docs/configuring_tor.md b/docs/configuring_tor.md index 442930d6e3..ecb03bfc32 100644 --- a/docs/configuring_tor.md +++ b/docs/configuring_tor.md @@ -1,25 +1,4 @@ -### Table of Contents -1. [Overview](#Overview)
-2. [Client-Only](#Client)
-2.1 [Description](#ClientDescription)
-2.2 [Command Line Example](#ClientCLIExample)
-2.3 [Config File Example](#ClientConfigFileExample)
-3. [Client-Server via Tor Hidden Service](#HiddenService)
-3.1 [Description](#HiddenServiceDescription)
-3.2 [Command Line Example](#HiddenServiceCLIExample)
-3.3 [Config File Example](#HiddenServiceConfigFileExample)
-4. [Bridge Mode (Not Anonymous)](#Bridge)
-4.1 [Description](#BridgeDescription)
-4.2 [Command Line Example](#BridgeCLIExample)
-4.3 [Config File Example](#BridgeConfigFileExample)
-5. [Tor Stream Isolation](#TorStreamIsolation)
-5.1 [Description](#TorStreamIsolationDescription)
-5.2 [Command Line Example](#TorStreamIsolationCLIExample)
-5.3 [Config File Example](#TorStreamIsolationFileExample)
- - - -### 1. Overview +# Configuring TOR btcd provides full support for anonymous networking via the [Tor Project](https://www.torproject.org/), including [client-only](#Client) @@ -34,13 +13,7 @@ network to run as both a client and a server so others may connect to you to as you are connecting to them. We recommend you take the time to setup a Tor hidden service for this reason. - - -### 2. Client-Only - - - -**2.1 Description**
+## Client-only Configuring btcd as a Tor client is straightforward. The first step is obviously to install Tor and ensure it is working. Once that is done, all that @@ -58,17 +31,13 @@ NOTE: Specifying the `--proxy` flag disables listening by default since you will not be reachable for inbound connections unless you also configure a Tor [hidden service](#HiddenService). - - -**2.2 Command Line Example**
+### Command line example ```bash -$ ./btcd --proxy=127.0.0.1:9050 +./btcd --proxy=127.0.0.1:9050 ``` - - -**2.3 Config File Example**
+### Config file example ```text [Application Options] @@ -76,13 +45,7 @@ $ ./btcd --proxy=127.0.0.1:9050 proxy=127.0.0.1:9050 ``` - - -### 3. Client-Server via Tor Hidden Service - - - -**3.1 Description**
+## Client-server via Tor hidden service The first step is to configure Tor to provide a hidden service. Documentation for this can be found on the Tor project website @@ -103,23 +66,20 @@ HiddenServicePort 8333 127.0.0.1:8333 Once Tor is configured to provide the hidden service and you have obtained your generated .onion address, configuring btcd as a Tor hidden service requires three flags: + * `--proxy` to identify the Tor (SOCKS 5) proxy to use for outgoing traffic. This is typically 127.0.0.1:9050. * `--listen` to enable listening for inbound connections since `--proxy` disables listening by default * `--externalip` to set the .onion address that is advertised to other peers - - -**3.2 Command Line Example**
+### Command line example ```bash -$ ./btcd --proxy=127.0.0.1:9050 --listen=127.0.0.1 --externalip=fooanon.onion +./btcd --proxy=127.0.0.1:9050 --listen=127.0.0.1 --externalip=fooanon.onion ``` - - -**3.3 Config File Example**
+### Config file example ```text [Application Options] @@ -129,13 +89,7 @@ listen=127.0.0.1 externalip=fooanon.onion ``` - - -### 4. Bridge Mode (Not Anonymous) - - - -**4.1 Description**
+## Bridge mode (not anonymous) btcd provides support for operating as a bridge between regular nodes and hidden service nodes. In particular this means only traffic which is directed to or @@ -154,17 +108,13 @@ mode, you only need to specify your hidden service's .onion address via the `--externalip` flag since traffic to and from .onion addresses are already routed via Tor due to the `--onion` flag. - - -**4.2 Command Line Example**
+### Command line example ```bash -$ ./btcd --onion=127.0.0.1:9050 --externalip=fooanon.onion +./btcd --onion=127.0.0.1:9050 --externalip=fooanon.onion ``` - - -**4.3 Config File Example**
+### Config file example ```text [Application Options] @@ -173,13 +123,7 @@ onion=127.0.0.1:9050 externalip=fooanon.onion ``` - - -### 5. Tor Stream Isolation - - - -**5.1 Description**
+## Tor stream isolation Tor stream isolation forces Tor to build a new circuit for each connection making it harder to correlate connections. @@ -187,17 +131,13 @@ making it harder to correlate connections. btcd provides support for Tor stream isolation by using the `--torisolation` flag. This option requires --proxy or --onionproxy to be set. - - -**5.2 Command Line Example**
+### Command line example ```bash -$ ./btcd --proxy=127.0.0.1:9050 --torisolation +./btcd --proxy=127.0.0.1:9050 --torisolation ``` - - -**5.3 Config File Example**
+### Config file example ```text [Application Options] diff --git a/docs/contact.md b/docs/contact.md new file mode 100644 index 0000000000..88b425e8bc --- /dev/null +++ b/docs/contact.md @@ -0,0 +1,15 @@ +# Contact + +## IRC + +* [irc.freenode.net](irc://irc.freenode.net), channel `#btcd` + +## Mailing Lists + +* [btcd](mailto:btcd+subscribe@opensource.conformal.com): discussion of btcd and its packages. +* [btcd-commits](mailto:btcd-commits+subscribe@opensource.conformal.com): readonly mail-out of source code changes. + +## Issue Tracker + +The [integrated github issue tracker](https://github.com/btcsuite/btcd/issues) +is used for this project. diff --git a/docs/controlling.md b/docs/controlling.md new file mode 100644 index 0000000000..93ab403b2e --- /dev/null +++ b/docs/controlling.md @@ -0,0 +1,34 @@ +# Controlling and querying btcd via btcctl + +btcctl is a command line utility that can be used to both control and query btcd +via [RPC](http://www.wikipedia.org/wiki/Remote_procedure_call). btcd does +**not** enable its RPC server by default; You must configure at minimum both an +RPC username and password or both an RPC limited username and password: + +* btcd.conf configuration file + +```bash +[Application Options] +rpcuser=myuser +rpcpass=SomeDecentp4ssw0rd +rpclimituser=mylimituser +rpclimitpass=Limitedp4ssw0rd +``` + +* btcctl.conf configuration file + +```bash +[Application Options] +rpcuser=myuser +rpcpass=SomeDecentp4ssw0rd +``` + +OR + +```bash +[Application Options] +rpclimituser=mylimituser +rpclimitpass=Limitedp4ssw0rd +``` + +For a list of available options, run: `$ btcctl --help` diff --git a/docs/default_ports.md b/docs/default_ports.md deleted file mode 100644 index 14e4eea2a7..0000000000 --- a/docs/default_ports.md +++ /dev/null @@ -1,15 +0,0 @@ -While btcd is highly configurable when it comes to the network configuration, -the following is intended to be a quick reference for the default ports used so -port forwarding can be configured as required. - -btcd provides a `--upnp` flag which can be used to automatically map the bitcoin -peer-to-peer listening port if your router supports UPnP. If your router does -not support UPnP, or you don't wish to use it, please note that only the bitcoin -peer-to-peer port should be forwarded unless you specifically want to allow RPC -access to your btcd from external sources such as in more advanced network -configurations. - -|Name|Port| -|----|----| -|Default Bitcoin peer-to-peer port|TCP 8333| -|Default RPC port|TCP 8334| diff --git a/docs/developer_resources.md b/docs/developer_resources.md new file mode 100644 index 0000000000..cec8ce9972 --- /dev/null +++ b/docs/developer_resources.md @@ -0,0 +1,37 @@ +# Developer Resources + +* [Code Contribution Guidelines](https://github.com/btcsuite/btcd/tree/master/docs/code_contribution_guidelines.md) + +* [JSON-RPC Reference](https://github.com/btcsuite/btcd/tree/master/docs/json_rpc_api.md) + * [RPC Examples](https://github.com/btcsuite/btcd/tree/master/docs/json_rpc_api.md#ExampleCode) + +* The btcsuite Bitcoin-related Go Packages: + * [btcrpcclient](https://github.com/btcsuite/btcd/tree/master/rpcclient) - Implements a + robust and easy to use Websocket-enabled Bitcoin JSON-RPC client + * [btcjson](https://github.com/btcsuite/btcd/tree/master/btcjson) - Provides an extensive API + for the underlying JSON-RPC command and return values + * [wire](https://github.com/btcsuite/btcd/tree/master/wire) - Implements the + Bitcoin wire protocol + * [peer](https://github.com/btcsuite/btcd/tree/master/peer) - + Provides a common base for creating and managing Bitcoin network peers. + * [blockchain](https://github.com/btcsuite/btcd/tree/master/blockchain) - + Implements Bitcoin block handling and chain selection rules + * [blockchain/fullblocktests](https://github.com/btcsuite/btcd/tree/master/blockchain/fullblocktests) - + Provides a set of block tests for testing the consensus validation rules + * [txscript](https://github.com/btcsuite/btcd/tree/master/txscript) - + Implements the Bitcoin transaction scripting language + * [btcec](https://github.com/btcsuite/btcd/tree/master/btcec) - Implements + support for the elliptic curve cryptographic functions needed for the + Bitcoin scripts + * [database](https://github.com/btcsuite/btcd/tree/master/database) - + Provides a database interface for the Bitcoin block chain + * [mempool](https://github.com/btcsuite/btcd/tree/master/mempool) - + Package mempool provides a policy-enforced pool of unmined bitcoin + transactions. + * [btcutil](https://github.com/btcsuite/btcutil) - Provides Bitcoin-specific + convenience functions and types + * [chainhash](https://github.com/btcsuite/btcd/tree/master/chaincfg/chainhash) - + Provides a generic hash type and associated functions that allows the + specific hash algorithm to be abstracted. + * [connmgr](https://github.com/btcsuite/btcd/tree/master/connmgr) - + Package connmgr implements a generic Bitcoin network connection manager. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000000..9d980626d4 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,57 @@ +# btcd + +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) +[![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd) + +btcd is an alternative full node bitcoin implementation written in Go (golang). + +This project is currently under active development and is in a Beta state. It +is extremely stable and has been in production use since October 2013. + +It properly downloads, validates, and serves the block chain using the exact +rules (including consensus bugs) for block acceptance as Bitcoin Core. We have +taken great care to avoid btcd causing a fork to the block chain. It includes a +full block validation testing framework which contains all of the 'official' +block acceptance tests (and some additional ones) that is run on every pull +request to help ensure it properly follows consensus. Also, it passes all of +the JSON test data in the Bitcoin Core code. + +It also properly relays newly mined blocks, maintains a transaction pool, and +relays individual transactions that have not yet made it into a block. It +ensures all individual transactions admitted to the pool follow the rules +required by the block chain and also includes more strict checks which filter +transactions based on miner requirements ("standard" transactions). + +One key difference between btcd and Bitcoin Core is that btcd does *NOT* include +wallet functionality and this was a very intentional design decision. See the +blog entry [here](https://web.archive.org/web/20171125143919/https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon) +for more details. This means you can't actually make or receive payments +directly with btcd. That functionality is provided by the +[btcwallet](https://github.com/btcsuite/btcwallet) and +[Paymetheus](https://github.com/btcsuite/Paymetheus) (Windows-only) projects +which are both under active development. + +## Documentation + +Documentation is a work-in-progress. It is available at [btcd.readthedocs.io](https://btcd.readthedocs.io). + +## Contents + +* [Installation](installation.md) +* [Update](update.md) +* [Configuration](configuration.md) +* [Configuring TOR](configuring_tor.md) +* [Docker](using_docker.md) +* [Controlling](controlling.md) +* [Mining](mining.md) +* [Wallet](wallet.md) +* [Developer resources](developer_resources.md) +* [JSON RPC API](json_rpc_api.md) +* [Code contribution guidelines](code_contribution_guidelines.md) +* [Contact](contact.md) + +## License + +btcd is licensed under the [copyfree](http://copyfree.org) ISC License. + diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000000..c3c206060b --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,76 @@ +# Installation + +The first step is to install btcd. See one of the following sections for +details on how to install on the supported operating systems. + +## Requirements + +[Go](http://golang.org) 1.11 or newer. + +## GPG Verification Key + +All official release tags are signed by Conformal so users can ensure the code +has not been tampered with and is coming from the btcsuite developers. To +verify the signature perform the following: + +* Download the Conformal public key: + https://raw.githubusercontent.com/btcsuite/btcd/master/release/GIT-GPG-KEY-conformal.txt + +* Import the public key into your GPG keyring: + + ```bash + gpg --import GIT-GPG-KEY-conformal.txt + ``` + +* Verify the release tag with the following command where `TAG_NAME` is a + placeholder for the specific tag: + + ```bash + git tag -v TAG_NAME + ``` + +## Windows Installation + +* Install the MSI available at: [btcd windows installer](https://github.com/btcsuite/btcd/releases) +* Launch btcd from the Start Menu + +## Linux/BSD/MacOSX/POSIX Installation + +* Install Go according to the [installation instructions](http://golang.org/doc/install) +* Ensure Go was installed properly and is a supported version: + +```bash +go version +go env GOROOT GOPATH +``` + +NOTE: The `GOROOT` and `GOPATH` above must not be the same path. It is +recommended that `GOPATH` is set to a directory in your home directory such as +`~/goprojects` to avoid write permission issues. It is also recommended to add +`$GOPATH/bin` to your `PATH` at this point. + +* Run the following commands to obtain btcd, all dependencies, and install it: + +```bash +git clone https://github.com/btcsuite/btcd $GOPATH/src/github.com/btcsuite/btcd +cd $GOPATH/src/github.com/btcsuite/btcd +GO111MODULE=on go install -v . ./cmd/... +``` + +* btcd (and utilities) will now be installed in ```$GOPATH/bin```. If you did + not already add the bin directory to your system path during Go installation, + we recommend you do so now. + +## Gentoo Linux Installation + +* [Install Layman](https://gitlab.com/bitcoin/gentoo) and enable the Bitcoin overlay. +* Copy or symlink `/var/lib/layman/bitcoin/Documentation/package.keywords/btcd-live` to `/etc/portage/package.keywords/` +* Install btcd: `$ emerge net-p2p/btcd` + +## Startup + +Typically btcd will run and start downloading the block chain with no extra +configuration necessary, however, there is an optional method to use a +`bootstrap.dat` file that may speed up the initial block chain download process. + +* [Using bootstrap.dat](https://github.com/btcsuite/btcd/blob/master/docs/configuration.md#using-bootstrapdat) diff --git a/docs/json_rpc_api.md b/docs/json_rpc_api.md index 61884df0a0..db292d2ba2 100644 --- a/docs/json_rpc_api.md +++ b/docs/json_rpc_api.md @@ -1,4 +1,5 @@ -### Table of Contents +# JSON RPC API + 1. [Overview](#Overview)
2. [HTTP POST Versus Websockets](#HttpPostVsWebsockets)
3. [Authentication](#Authentication)
diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 0000000000..922152e96a --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=. +set BUILDDIR=_build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/mining.md b/docs/mining.md new file mode 100644 index 0000000000..29a3e89858 --- /dev/null +++ b/docs/mining.md @@ -0,0 +1,30 @@ +# Mining + +btcd supports the `getblocktemplate` RPC. +The limited user cannot access this RPC. + +## Add the payment addresses with the `miningaddr` option + +```bash +[Application Options] +rpcuser=myuser +rpcpass=SomeDecentp4ssw0rd +miningaddr=12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX +miningaddr=1M83ju3EChKYyysmM2FXtLNftbacagd8FR +``` + +## Add btcd's RPC TLS certificate to system Certificate Authority list + +`cgminer` uses [curl](http://curl.haxx.se/) to fetch data from the RPC server. +Since curl validates the certificate by default, we must install the `btcd` RPC +certificate into the default system Certificate Authority list. + +## Ubuntu + +1. Copy rpc.cert to /usr/share/ca-certificates: `# cp /home/user/.btcd/rpc.cert /usr/share/ca-certificates/btcd.crt` +2. Add btcd.crt to /etc/ca-certificates.conf: `# echo btcd.crt >> /etc/ca-certificates.conf` +3. Update the CA certificate list: `# update-ca-certificates` + +## Set your mining software url to use https + +`cgminer -o https://127.0.0.1:8334 -u rpcuser -p rpcpassword` diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 0000000000..e16dd4b38f --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1 @@ +sphinx_markdown_tables diff --git a/docs/table_of_content.md b/docs/table_of_content.md new file mode 100644 index 0000000000..85f08a97b3 --- /dev/null +++ b/docs/table_of_content.md @@ -0,0 +1,13 @@ +# Contents + +* [Installation](installation.md) +* [Update](update.md) +* [Configuration](configuration.md) +* [Configuring TOR](configuring_tor.md) +* [Controlling](controlling.md) +* [Mining](mining.md) +* [Wallet](wallet.md) +* [Developer resources](developer_resources.md) +* [JSON RPC API](json_rpc_api.md) +* [Code contribution guidelines](code_contribution_guidelines.md) +* [Contact](contact.md) diff --git a/docs/update.md b/docs/update.md new file mode 100644 index 0000000000..1fb847cf9f --- /dev/null +++ b/docs/update.md @@ -0,0 +1,8 @@ +# Update + +* Run the following commands to update btcd, all dependencies, and install it: + +```bash +cd $GOPATH/src/github.com/btcsuite/btcd +git pull && GO111MODULE=on go install -v . ./cmd/... +``` diff --git a/docs/using_bootstrap_dat.md b/docs/using_bootstrap_dat.md deleted file mode 100644 index de7e08c0ce..0000000000 --- a/docs/using_bootstrap_dat.md +++ /dev/null @@ -1,79 +0,0 @@ -### Table of Contents -1. [What is bootstrap.dat?](#What)
-2. [What are the pros and cons of using bootstrap.dat?](#ProsCons) -3. [Where do I get bootstrap.dat?](#Obtaining) -4. [How do I know I can trust the bootstrap.dat I downloaded?](#Trust) -5. [How do I use bootstrap.dat with btcd?](#Importing) - - - -### 1. What is bootstrap.dat? - -It is a flat, binary file containing bitcoin blockchain data starting from the -genesis block and continuing through a relatively recent block height depending -on the last time it was updated. - -See [this](https://bitcointalk.org/index.php?topic=145386.0) thread on -bitcointalk for more details. - -**NOTE:** Using bootstrap.dat is entirely optional. Btcd will download the -block chain from other peers through the Bitcoin protocol with no extra -configuration needed. - - - -### 2. What are the pros and cons of using bootstrap.dat? - -Pros: -- Typically accelerates the initial process of bringing up a new node as it - downloads from public P2P nodes and generally is able to achieve faster - download speeds -- It is particularly beneficial when bringing up multiple nodes as you only need - to download the data once - -Cons: -- Requires you to setup and configure a torrent client if you don't already have - one available -- Requires roughly twice as much disk space since you'll need the flat file as - well as the imported database - - - -### 3. Where do I get bootstrap.dat? - -The bootstrap.dat file is made available via a torrent. See -[this](https://bitcointalk.org/index.php?topic=145386.0) thread on bitcointalk -for the torrent download details. - - - -### 4. How do I know I can trust the bootstrap.dat I downloaded? - -You don't need to trust the file as the `addblock` utility verifies every block -using the same rules that are used when downloading the block chain normally -through the Bitcoin protocol. Additionally, the chain rules contain hard-coded -checkpoints for the known-good block chain at periodic intervals. This ensures -that not only is it a valid chain, but it is the same chain that everyone else -is using. - - - -### 5. How do I use bootstrap.dat with btcd? - -btcd comes with a separate utility named `addblock` which can be used to import -`bootstrap.dat`. This approach is used since the import is a one-time operation -and we prefer to keep the daemon itself as lightweight as possible. - -1. Stop btcd if it is already running. This is required since addblock needs to - access the database used by btcd and it will be locked if btcd is using it. -2. Note the path to the downloaded bootstrap.dat file. -3. Run the addblock utility with the `-i` argument pointing to the location of - boostrap.dat:

-**Windows:** -```bat -C:\> "%PROGRAMFILES%\Btcd Suite\Btcd\addblock" -i C:\Path\To\bootstrap.dat -``` -**Linux/Unix/BSD/POSIX:** -```bash -$ $GOPATH/bin/addblock -i /path/to/bootstrap.dat -``` diff --git a/docs/using_docker.md b/docs/using_docker.md new file mode 100644 index 0000000000..0809abc1c8 --- /dev/null +++ b/docs/using_docker.md @@ -0,0 +1,160 @@ +# Using Docker + +- [Using Docker](#using-docker) + - [Introduction](#introduction) + - [Docker volumes](#docker-volumes) + - [Known error messages when starting the btcd container](#known-error-messages-when-starting-the-btcd-container) + - [Examples](#examples) + - [Preamble](#preamble) + - [Full node without RPC port](#full-node-without-rpc-port) + - [Full node with RPC port](#full-node-with-rpc-port) + - [Full node with RPC port running on TESTNET](#full-node-with-rpc-port-running-on-testnet) + +## Introduction + +With Docker you can easily set up *btcd* to run your Bitcoin full node. You can find the official *btcd* Docker images on Docker Hub [btcsuite/btcd](https://hub.docker.com/r/btcsuite/btcd). The Docker source file of this image is located at [Dockerfile](https://github.com/btcsuite/btcd/blob/master/Dockerfile). + +This documentation focuses on running Docker container with *docker-compose.yml* files. These files are better to read and you can use them as a template for your own use. For more information about Docker and Docker compose visit the official [Docker documentation](https://docs.docker.com/). + +## Docker volumes + +**Special diskspace hint**: The following examples are using a Docker managed volume. The volume is named *btcd-data* This will use a lot of disk space, because it contains the full Bitcoin blockchain. Please make yourself familiar with [Docker volumes](https://docs.docker.com/storage/volumes/). + +The *btcd-data* volume will be reused, if you upgrade your *docker-compose.yml* file. Keep in mind, that it is not automatically removed by Docker, if you delete the btcd container. If you don't need the volume anymore, please delete it manually with the command: + +```bash +docker volume ls +docker volume rm btcd-data +``` + +For binding a local folder to your *btcd* container please read the [Docker documentation](https://docs.docker.com/). The preferred way is to use a Docker managed volume. + +## Known error messages when starting the btcd container + +We pass all needed arguments to *btcd* as command line parameters in our *docker-compose.yml* file. It doesn't make sense to create a *btcd.conf* file. This would make things too complicated. Anyhow *btcd* will complain with following log messages when starting. These messages can be ignored: + +```bash +Error creating a default config file: open /sample-btcd.conf: no such file or directory +... +[WRN] BTCD: open /root/.btcd/btcd.conf: no such file or directory +``` + +## Examples + +### Preamble + +All following examples uses some defaults: + +- container_name: btcd + Name of the docker container that is be shown by e.g. ```docker ps -a``` + +- hostname: btcd **(very important to set a fixed name before first start)** + The internal hostname in the docker container. By default, docker is recreating the hostname every time you change the *docker-compose.yml* file. The default hostnames look like *ef00548d4fa5*. This is a problem when using the *btcd* RPC port. The RPC port is using a certificate to validate the hostname. If the hostname changes you need to recreate the certificate. To avoid this, you should set a fixed hostname before the first start. This ensures, that the docker volume is created with a certificate with this hostname. + +- restart: unless-stopped + Starts the *btcd* container when Docker starts, except that when the container is stopped (manually or otherwise), it is not restarted even after Docker restarts. + +To use the following examples create an empty directory. In this directory create a file named *docker-compose.yml*, copy and paste the example into the *docker-compose.yml* file and run it. + +```bash +mkdir ~/btcd-docker +cd ~/btcd-docker +touch docker-compose.yaml +nano docker-compose.yaml (use your favourite editor to edit the compose file) +docker-compose up (creates and starts a new btcd container) +``` + +With the following commands you can control *docker-compose*: + +```docker-compose up -d``` (creates and starts the container in background) + +```docker-compose down``` (stops and delete the container. **The docker volume btcd-data will not be deleted**) + +```docker-compose stop``` (stops the container) + +```docker-compose start``` (starts the container) + +```docker ps -a``` (list all running and stopped container) + +```docker volume ls``` (lists all docker volumes) + +```docker logs btcd``` (shows the log ) + +```docker-compose help``` (brings up some helpful information) + +### Full node without RPC port + +Let's start with an easy example. If you just want to create a full node without the need of using the RPC port, you can use the following example. This example will launch *btcd* and exposes only the default p2p port 8333 to the outside world: + +```yaml +version: "2" + +services: + btcd: + container_name: btcd + hostname: btcd + image: btcsuite/btcd:latest + restart: unless-stopped + volumes: + - btcd-data:/root/.btcd + ports: + - 8333:8333 + +volumes: + btcd-data: +``` + +### Full node with RPC port + +To use the RPC port of *btcd* you need to specify a *username* and a very strong *password*. If you want to connect to the RPC port from the internet, you need to expose port 8334(RPC) as well. + +```yaml +version: "2" + +services: + btcd: + container_name: btcd + hostname: btcd + image: btcsuite/btcd:latest + restart: unless-stopped + volumes: + - btcd-data:/root/.btcd + ports: + - 8333:8333 + - 8334:8334 + command: [ + "--rpcuser=[CHOOSE_A_USERNAME]", + "--rpcpass=[CREATE_A_VERY_HARD_PASSWORD]" + ] + +volumes: + btcd-data: +``` + +### Full node with RPC port running on TESTNET + +To run a node on testnet, you need to provide the *--testnet* argument. The ports for testnet are 18333 (p2p) and 18334 (RPC): + +```yaml +version: "2" + +services: + btcd: + container_name: btcd + hostname: btcd + image: btcsuite/btcd:latest + restart: unless-stopped + volumes: + - btcd-data:/root/.btcd + ports: + - 18333:18333 + - 18334:18334 + command: [ + "--testnet", + "--rpcuser=[CHOOSE_A_USERNAME]", + "--rpcpass=[CREATE_A_VERY_HARD_PASSWORD]" + ] + +volumes: + btcd-data: +``` diff --git a/docs/wallet.md b/docs/wallet.md new file mode 100644 index 0000000000..cc123aa7d5 --- /dev/null +++ b/docs/wallet.md @@ -0,0 +1,5 @@ +# Wallet + +btcd was intentionally developed without an integrated wallet for security +reasons. Please see [btcwallet](https://github.com/btcsuite/btcwallet) for more +information. diff --git a/go.mod b/go.mod index 582ceefe65..049b97fe6c 100644 --- a/go.mod +++ b/go.mod @@ -1,10 +1,8 @@ module github.com/btcsuite/btcd -go 1.14 - require ( github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f - github.com/btcsuite/btcutil v1.0.2 + github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd github.com/btcsuite/goleveldb v1.0.0 github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792 @@ -15,3 +13,5 @@ require ( github.com/jrick/logrotate v1.0.0 golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 ) + +go 1.14 diff --git a/go.sum b/go.sum index 392d70f1b7..e259d0ec77 100644 --- a/go.sum +++ b/go.sum @@ -5,8 +5,8 @@ github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f h1:bAs4lUbRJpnnkd9 github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA= github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d h1:yJzD/yFppdVCf6ApMkVy8cUxV0XrxdP9rVf6D87/Mng= github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg= -github.com/btcsuite/btcutil v1.0.2 h1:9iZ1Terx9fMIOtq1VrwdqfsATL9MC2l8ZrUY6YZ2uts= -github.com/btcsuite/btcutil v1.0.2/go.mod h1:j9HUFwoQRsZL3V4n+qG+CUnEGHOarIxfC3Le2Yhbcts= +github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce h1:YtWJF7RHm2pYCvA5t0RPmAaLUhREsKuKd+SLhxFbFeQ= +github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce/go.mod h1:0DVlHczLPewLcPGEIeUEzfOJhqGPQ0mJJRDBtD307+o= github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd h1:R/opQEbFEy9JGkIguV40SvRY1uliPX8ifOvi6ICsFCw= github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg= github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd h1:qdGvebPBDuYDPGi1WCPjy1tGyMpmDK8IEapSsszn7HE= @@ -22,6 +22,7 @@ github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtE github.com/btcsuite/winsvc v1.0.0 h1:J9B4L7e3oqhXOcm+2IuNApwzQec85lE+QaikUcCs+dk= github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs= github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/decred/dcrd/lru v1.0.0 h1:Kbsb1SFDsIlaupWPwsPp+dkxiBY1frcS07PCPgotKz8= diff --git a/goclean.sh b/goclean.sh index bb0cd7b600..dad9f8f1dc 100755 --- a/goclean.sh +++ b/goclean.sh @@ -4,11 +4,12 @@ # 3. go vet (http://golang.org/cmd/vet) # 4. gosimple (https://github.com/dominikh/go-simple) # 5. unconvert (https://github.com/mdempsky/unconvert) -# +# 6. race detector (http://blog.golang.org/race-detector) +# 7. test coverage (http://blog.golang.org/cover) set -ex -go test -tags="rpctest" ./... +env GORACE="halt_on_error=1" go test -race -tags="rpctest" -covermode atomic -coverprofile=profile.cov ./... # Automatic checks golangci-lint run --deadline=10m --disable-all \ diff --git a/integration/README.md b/integration/README.md index 52bbf51167..5f6f14eace 100644 --- a/integration/README.md +++ b/integration/README.md @@ -1,7 +1,7 @@ integration =========== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) This contains integration tests which make use of the diff --git a/integration/bip0009_test.go b/integration/bip0009_test.go index df3721b1e6..9bdec34fbb 100644 --- a/integration/bip0009_test.go +++ b/integration/bip0009_test.go @@ -33,7 +33,7 @@ const ( // ensures its version either has the provided bit set or unset per the set // flag. func assertVersionBit(r *rpctest.Harness, t *testing.T, hash *chainhash.Hash, bit uint8, set bool) { - block, err := r.Node.GetBlock(hash) + block, err := r.Client.GetBlock(hash) if err != nil { t.Fatalf("failed to retrieve block %v: %v", hash, err) } @@ -53,7 +53,7 @@ func assertVersionBit(r *rpctest.Harness, t *testing.T, hash *chainhash.Hash, bi // assertChainHeight retrieves the current chain height from the given test // harness and ensures it matches the provided expected height. func assertChainHeight(r *rpctest.Harness, t *testing.T, expectedHeight uint32) { - height, err := r.Node.GetBlockCount() + height, err := r.Client.GetBlockCount() if err != nil { t.Fatalf("failed to retrieve block height: %v", err) } @@ -96,7 +96,7 @@ func assertSoftForkStatus(r *rpctest.Harness, t *testing.T, forkKey string, stat "threshold state %v to string", line, state) } - info, err := r.Node.GetBlockChainInfo() + info, err := r.Client.GetBlockChainInfo() if err != nil { t.Fatalf("failed to retrieve chain info: %v", err) } @@ -129,7 +129,7 @@ func assertSoftForkStatus(r *rpctest.Harness, t *testing.T, forkKey string, stat // specific soft fork deployment to test. func testBIP0009(t *testing.T, forkKey string, deploymentID uint32) { // Initialize the primary mining node with only the genesis block. - r, err := rpctest.New(&chaincfg.RegressionNetParams, nil, nil) + r, err := rpctest.New(&chaincfg.RegressionNetParams, nil, nil, "") if err != nil { t.Fatalf("unable to create primary harness: %v", err) } @@ -320,7 +320,7 @@ func TestBIP0009Mining(t *testing.T) { t.Parallel() // Initialize the primary mining node with only the genesis block. - r, err := rpctest.New(&chaincfg.SimNetParams, nil, nil) + r, err := rpctest.New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatalf("unable to create primary harness: %v", err) } @@ -339,7 +339,7 @@ func TestBIP0009Mining(t *testing.T) { // in the defined threshold state. deployment := &r.ActiveNet.Deployments[chaincfg.DeploymentTestDummy] testDummyBitNum := deployment.BitNumber - hashes, err := r.Node.Generate(1) + hashes, err := r.Client.Generate(1) if err != nil { t.Fatalf("unable to generate blocks: %v", err) } @@ -358,7 +358,7 @@ func TestBIP0009Mining(t *testing.T) { // dummy deployment as started. confirmationWindow := r.ActiveNet.MinerConfirmationWindow numNeeded := confirmationWindow - 1 - hashes, err = r.Node.Generate(numNeeded) + hashes, err = r.Client.Generate(numNeeded) if err != nil { t.Fatalf("failed to generated %d blocks: %v", numNeeded, err) } @@ -373,7 +373,7 @@ func TestBIP0009Mining(t *testing.T) { // The last generated block should still have the test bit set in the // version since the btcd mining code will have recognized the test // dummy deployment as locked in. - hashes, err = r.Node.Generate(confirmationWindow) + hashes, err = r.Client.Generate(confirmationWindow) if err != nil { t.Fatalf("failed to generated %d blocks: %v", confirmationWindow, err) @@ -392,7 +392,7 @@ func TestBIP0009Mining(t *testing.T) { // version since the btcd mining code will have recognized the test // dummy deployment as activated and thus there is no longer any need // to set the bit. - hashes, err = r.Node.Generate(confirmationWindow) + hashes, err = r.Client.Generate(confirmationWindow) if err != nil { t.Fatalf("failed to generated %d blocks: %v", confirmationWindow, err) diff --git a/integration/csv_fork_test.go b/integration/csv_fork_test.go index 345217c864..3146634966 100644 --- a/integration/csv_fork_test.go +++ b/integration/csv_fork_test.go @@ -57,14 +57,14 @@ func makeTestOutput(r *rpctest.Harness, t *testing.T, if err != nil { return nil, nil, nil, err } - txHash, err := r.Node.SendRawTransaction(fundTx, true) + txHash, err := r.Client.SendRawTransaction(fundTx, true) if err != nil { return nil, nil, nil, err } // The transaction created above should be included within the next // generated block. - blockHash, err := r.Node.Generate(1) + blockHash, err := r.Client.Generate(1) if err != nil { return nil, nil, nil, err } @@ -109,7 +109,7 @@ func TestBIP0113Activation(t *testing.T) { t.Parallel() btcdCfg := []string{"--rejectnonstd"} - r, err := rpctest.New(&chaincfg.SimNetParams, nil, btcdCfg) + r, err := rpctest.New(&chaincfg.SimNetParams, nil, btcdCfg, "") if err != nil { t.Fatal("unable to create primary harness: ", err) } @@ -151,7 +151,7 @@ func TestBIP0113Activation(t *testing.T) { // We set the lock-time of the transaction to just one minute after the // current MTP of the chain. - chainInfo, err := r.Node.GetBlockChainInfo() + chainInfo, err := r.Client.GetBlockChainInfo() if err != nil { t.Fatalf("unable to query for chain info: %v", err) } @@ -167,7 +167,7 @@ func TestBIP0113Activation(t *testing.T) { // This transaction should be rejected from the mempool as using MTP // for transactions finality is now a policy rule. Additionally, the // exact error should be the rejection of a non-final transaction. - _, err = r.Node.SendRawTransaction(tx, true) + _, err = r.Client.SendRawTransaction(tx, true) if err == nil { t.Fatalf("transaction accepted, but should be non-final") } else if !strings.Contains(err.Error(), "not finalized") { @@ -201,7 +201,7 @@ func TestBIP0113Activation(t *testing.T) { // height 299. The getblockchaininfo call checks the state for the // block AFTER the current height. numBlocks := (r.ActiveNet.MinerConfirmationWindow * 2) - 4 - if _, err := r.Node.Generate(numBlocks); err != nil { + if _, err := r.Client.Generate(numBlocks); err != nil { t.Fatalf("unable to generate blocks: %v", err) } @@ -220,7 +220,7 @@ func TestBIP0113Activation(t *testing.T) { // rejected. timeLockDeltas := []int64{-1, 0, 1} for _, timeLockDelta := range timeLockDeltas { - chainInfo, err = r.Node.GetBlockChainInfo() + chainInfo, err = r.Client.GetBlockChainInfo() if err != nil { t.Fatalf("unable to query for chain info: %v", err) } @@ -257,7 +257,7 @@ func TestBIP0113Activation(t *testing.T) { // accepted as it has a lock-time of one // second _before_ the current MTP. - _, err = r.Node.SendRawTransaction(tx, true) + _, err = r.Client.SendRawTransaction(tx, true) if err == nil && timeLockDelta >= 0 { t.Fatal("transaction was accepted into the mempool " + "but should be rejected!") @@ -366,7 +366,7 @@ func spendCSVOutput(redeemScript []byte, csvUTXO *wire.OutPoint, func assertTxInBlock(r *rpctest.Harness, t *testing.T, blockHash *chainhash.Hash, txid *chainhash.Hash) { - block, err := r.Node.GetBlock(blockHash) + block, err := r.Client.GetBlock(blockHash) if err != nil { t.Fatalf("unable to get block: %v", err) } @@ -405,7 +405,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { // relative lock times. btcdCfg := []string{"--rejectnonstd"} - r, err := rpctest.New(&chaincfg.SimNetParams, nil, btcdCfg) + r, err := rpctest.New(&chaincfg.SimNetParams, nil, btcdCfg, "") if err != nil { t.Fatal("unable to create primary harness: ", err) } @@ -449,10 +449,10 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { // As the transaction is p2sh it should be accepted into the // mempool and found within the next generated block. - if _, err := r.Node.SendRawTransaction(tx, true); err != nil { + if _, err := r.Client.SendRawTransaction(tx, true); err != nil { t.Fatalf("unable to broadcast tx: %v", err) } - blocks, err := r.Node.Generate(1) + blocks, err := r.Client.Generate(1) if err != nil { t.Fatalf("unable to generate blocks: %v", err) } @@ -469,7 +469,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { // This transaction should be rejected from the mempool since // CSV validation is already mempool policy pre-fork. - _, err = r.Node.SendRawTransaction(spendingTx, true) + _, err = r.Client.SendRawTransaction(spendingTx, true) if err == nil { t.Fatalf("transaction should have been rejected, but was " + "instead accepted") @@ -496,7 +496,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { // height 299. The getblockchaininfo call checks the state for the // block AFTER the current height. numBlocks := (r.ActiveNet.MinerConfirmationWindow * 2) - 8 - if _, err := r.Node.Generate(numBlocks); err != nil { + if _, err := r.Client.Generate(numBlocks); err != nil { t.Fatalf("unable to generate blocks: %v", err) } @@ -530,7 +530,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { t.Fatalf("unable to create CSV output: %v", err) } - if _, err := r.Node.SendRawTransaction(tx, true); err != nil { + if _, err := r.Client.SendRawTransaction(tx, true); err != nil { t.Fatalf("unable to broadcast transaction: %v", err) } @@ -542,17 +542,17 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { } // Mine a single block including all the transactions generated above. - if _, err := r.Node.Generate(1); err != nil { + if _, err := r.Client.Generate(1); err != nil { t.Fatalf("unable to generate block: %v", err) } // Now mine 10 additional blocks giving the inputs generated above a // age of 11. Space out each block 10 minutes after the previous block. - prevBlockHash, err := r.Node.GetBestBlockHash() + prevBlockHash, err := r.Client.GetBestBlockHash() if err != nil { t.Fatalf("unable to get prior block hash: %v", err) } - prevBlock, err := r.Node.GetBlock(prevBlockHash) + prevBlock, err := r.Client.GetBlock(prevBlockHash) if err != nil { t.Fatalf("unable to get block: %v", err) } @@ -652,7 +652,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { } for i, test := range tests { - txid, err := r.Node.SendRawTransaction(test.tx, true) + txid, err := r.Client.SendRawTransaction(test.tx, true) switch { // Test case passes, nothing further to report. case test.accept && err == nil: @@ -686,7 +686,7 @@ func TestBIP0068AndBIP0112Activation(t *testing.T) { // Generate a block, the transaction should be included within // the newly mined block. - blockHashes, err := r.Node.Generate(1) + blockHashes, err := r.Client.Generate(1) if err != nil { t.Fatalf("unable to mine block: %v", err) } diff --git a/integration/rpcserver_test.go b/integration/rpcserver_test.go index df526442be..5875b35353 100644 --- a/integration/rpcserver_test.go +++ b/integration/rpcserver_test.go @@ -15,22 +15,24 @@ import ( "testing" "github.com/btcsuite/btcd/chaincfg" + "github.com/btcsuite/btcd/chaincfg/chainhash" "github.com/btcsuite/btcd/integration/rpctest" + "github.com/btcsuite/btcd/rpcclient" ) func testGetBestBlock(r *rpctest.Harness, t *testing.T) { - _, prevbestHeight, err := r.Node.GetBestBlock() + _, prevbestHeight, err := r.Client.GetBestBlock() if err != nil { t.Fatalf("Call to `getbestblock` failed: %v", err) } // Create a new block connecting to the current tip. - generatedBlockHashes, err := r.Node.Generate(1) + generatedBlockHashes, err := r.Client.Generate(1) if err != nil { t.Fatalf("Unable to generate block: %v", err) } - bestHash, bestHeight, err := r.Node.GetBestBlock() + bestHash, bestHeight, err := r.Client.GetBestBlock() if err != nil { t.Fatalf("Call to `getbestblock` failed: %v", err) } @@ -50,17 +52,17 @@ func testGetBestBlock(r *rpctest.Harness, t *testing.T) { func testGetBlockCount(r *rpctest.Harness, t *testing.T) { // Save the current count. - currentCount, err := r.Node.GetBlockCount() + currentCount, err := r.Client.GetBlockCount() if err != nil { t.Fatalf("Unable to get block count: %v", err) } - if _, err := r.Node.Generate(1); err != nil { + if _, err := r.Client.Generate(1); err != nil { t.Fatalf("Unable to generate block: %v", err) } // Count should have increased by one. - newCount, err := r.Node.GetBlockCount() + newCount, err := r.Client.GetBlockCount() if err != nil { t.Fatalf("Unable to get block count: %v", err) } @@ -72,17 +74,17 @@ func testGetBlockCount(r *rpctest.Harness, t *testing.T) { func testGetBlockHash(r *rpctest.Harness, t *testing.T) { // Create a new block connecting to the current tip. - generatedBlockHashes, err := r.Node.Generate(1) + generatedBlockHashes, err := r.Client.Generate(1) if err != nil { t.Fatalf("Unable to generate block: %v", err) } - info, err := r.Node.GetInfo() + info, err := r.Client.GetInfo() if err != nil { t.Fatalf("call to getinfo cailed: %v", err) } - blockHash, err := r.Node.GetBlockHash(int64(info.Blocks)) + blockHash, err := r.Client.GetBlockHash(int64(info.Blocks)) if err != nil { t.Fatalf("Call to `getblockhash` failed: %v", err) } @@ -94,10 +96,50 @@ func testGetBlockHash(r *rpctest.Harness, t *testing.T) { } } +func testBulkClient(r *rpctest.Harness, t *testing.T) { + // Create a new block connecting to the current tip. + generatedBlockHashes, err := r.Client.Generate(20) + if err != nil { + t.Fatalf("Unable to generate block: %v", err) + } + + var futureBlockResults []rpcclient.FutureGetBlockResult + for _, hash := range generatedBlockHashes { + futureBlockResults = append(futureBlockResults, r.BatchClient.GetBlockAsync(hash)) + } + + err = r.BatchClient.Send() + if err != nil { + t.Fatal(err) + } + + isKnownBlockHash := func(blockHash chainhash.Hash) bool { + for _, hash := range generatedBlockHashes { + if blockHash.IsEqual(hash) { + return true + } + } + return false + } + + for _, block := range futureBlockResults { + msgBlock, err := block.Receive() + if err != nil { + t.Fatal(err) + } + blockHash := msgBlock.Header.BlockHash() + if !isKnownBlockHash(blockHash) { + t.Fatalf("expected hash %s to be in generated hash list", blockHash) + } + } + +} + var rpcTestCases = []rpctest.HarnessTestCase{ testGetBestBlock, testGetBlockCount, testGetBlockHash, + testBulkClient, } var primaryHarness *rpctest.Harness @@ -109,7 +151,9 @@ func TestMain(m *testing.M) { // ensure that non-standard transactions aren't accepted into the // mempool or relayed. btcdCfg := []string{"--rejectnonstd"} - primaryHarness, err = rpctest.New(&chaincfg.SimNetParams, nil, btcdCfg) + primaryHarness, err = rpctest.New( + &chaincfg.SimNetParams, nil, btcdCfg, "", + ) if err != nil { fmt.Println("unable to create primary harness: ", err) os.Exit(1) diff --git a/integration/rpctest/README.md b/integration/rpctest/README.md index fceeed20a0..79f45bc857 100644 --- a/integration/rpctest/README.md +++ b/integration/rpctest/README.md @@ -1,9 +1,9 @@ rpctest ======= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/integration/rpctest) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/integration/rpctest) Package rpctest provides a btcd-specific RPC testing harness crafting and executing integration tests by driving a `btcd` instance via the `RPC` diff --git a/integration/rpctest/blockgen.go b/integration/rpctest/blockgen.go index de5821b0c1..0d802f5a48 100644 --- a/integration/rpctest/blockgen.go +++ b/integration/rpctest/blockgen.go @@ -14,6 +14,7 @@ import ( "github.com/btcsuite/btcd/blockchain" "github.com/btcsuite/btcd/chaincfg" "github.com/btcsuite/btcd/chaincfg/chainhash" + "github.com/btcsuite/btcd/mining" "github.com/btcsuite/btcd/txscript" "github.com/btcsuite/btcd/wire" "github.com/btcsuite/btcutil" @@ -181,6 +182,21 @@ func CreateBlock(prevBlock *btcutil.Block, inclusionTxs []*btcutil.Tx, if inclusionTxs != nil { blockTxns = append(blockTxns, inclusionTxs...) } + + // We must add the witness commitment to the coinbase if any + // transactions are segwit. + witnessIncluded := false + for i := 1; i < len(blockTxns); i++ { + if blockTxns[i].MsgTx().HasWitness() { + witnessIncluded = true + break + } + } + + if witnessIncluded { + _ = mining.AddWitnessCommitment(coinbaseTx, blockTxns) + } + merkles := blockchain.BuildMerkleTreeStore(blockTxns, false) var block wire.MsgBlock block.Header = wire.BlockHeader{ diff --git a/integration/rpctest/memwallet.go b/integration/rpctest/memwallet.go index f16130750a..59b0ef4c08 100644 --- a/integration/rpctest/memwallet.go +++ b/integration/rpctest/memwallet.go @@ -125,7 +125,7 @@ func newMemWallet(net *chaincfg.Params, harnessID uint32) (*memWallet, error) { // The first child key from the hd root is reserved as the coinbase // generation address. - coinbaseChild, err := hdRoot.Child(0) + coinbaseChild, err := hdRoot.Derive(0) if err != nil { return nil, err } @@ -337,7 +337,7 @@ func (m *memWallet) unwindBlock(update *chainUpdate) { func (m *memWallet) newAddress() (btcutil.Address, error) { index := m.hdIndex - childKey, err := m.hdRoot.Child(index) + childKey, err := m.hdRoot.Derive(index) if err != nil { return nil, err } @@ -509,7 +509,7 @@ func (m *memWallet) CreateTransaction(outputs []*wire.TxOut, outPoint := txIn.PreviousOutPoint utxo := m.utxos[outPoint] - extendedKey, err := m.hdRoot.Child(utxo.keyIndex) + extendedKey, err := m.hdRoot.Derive(utxo.keyIndex) if err != nil { return nil, err } diff --git a/integration/rpctest/node.go b/integration/rpctest/node.go index 6aec2b1168..73dc15fca9 100644 --- a/integration/rpctest/node.go +++ b/integration/rpctest/node.go @@ -41,10 +41,18 @@ type nodeConfig struct { } // newConfig returns a newConfig with all default values. -func newConfig(prefix, certFile, keyFile string, extra []string) (*nodeConfig, error) { - btcdPath, err := btcdExecutablePath() - if err != nil { - btcdPath = "btcd" +func newConfig(prefix, certFile, keyFile string, extra []string, + customExePath string) (*nodeConfig, error) { + + var btcdPath string + if customExePath != "" { + btcdPath = customExePath + } else { + var err error + btcdPath, err = btcdExecutablePath() + if err != nil { + btcdPath = "btcd" + } } a := &nodeConfig{ diff --git a/integration/rpctest/rpc_harness.go b/integration/rpctest/rpc_harness.go index 1c2612e47b..679ae4e478 100644 --- a/integration/rpctest/rpc_harness.go +++ b/integration/rpctest/rpc_harness.go @@ -7,6 +7,7 @@ package rpctest import ( "fmt" "io/ioutil" + "math/rand" "net" "os" "path/filepath" @@ -34,22 +35,20 @@ const ( // BlockVersion is the default block version used when generating // blocks. BlockVersion = 4 + + // DefaultMaxConnectionRetries is the default number of times we re-try + // to connect to the node after starting it. + DefaultMaxConnectionRetries = 20 + + // DefaultConnectionRetryTimeout is the default duration we wait between + // two connection attempts. + DefaultConnectionRetryTimeout = 50 * time.Millisecond ) var ( // current number of active test nodes. numTestInstances = 0 - // processID is the process ID of the current running process. It is - // used to calculate ports based upon it when launching an rpc - // harnesses. The intent is to allow multiple process to run in - // parallel without port collisions. - // - // It should be noted however that there is still some small probability - // that there will be port collisions either due to other processes - // running or simply due to the stars aligning on the process IDs. - processID = os.Getpid() - // testInstances is a private package-level slice used to keep track of // all active test harnesses. This global can be used to perform // various "joins", shutdown several active harnesses after a test, @@ -58,6 +57,13 @@ var ( // Used to protest concurrent access to above declared variables. harnessStateMtx sync.RWMutex + + // ListenAddressGenerator is a function that is used to generate two + // listen addresses (host:port), one for the P2P listener and one for + // the RPC listener. This is exported to allow overwriting of the + // default behavior which isn't very concurrency safe (just selecting + // a random port can produce collisions and therefore flakes). + ListenAddressGenerator = generateListeningAddresses ) // HarnessTestCase represents a test-case which utilizes an instance of the @@ -78,15 +84,23 @@ type Harness struct { // to. ActiveNet *chaincfg.Params - Node *rpcclient.Client - node *node - handlers *rpcclient.NotificationHandlers + // MaxConnRetries is the maximum number of times we re-try to connect to + // the node after starting it. + MaxConnRetries int + + // ConnectionRetryTimeout is the duration we wait between two connection + // attempts. + ConnectionRetryTimeout time.Duration + + Client *rpcclient.Client + BatchClient *rpcclient.Client + node *node + handlers *rpcclient.NotificationHandlers wallet *memWallet - testNodeDir string - maxConnRetries int - nodeNum int + testNodeDir string + nodeNum int sync.Mutex } @@ -94,11 +108,12 @@ type Harness struct { // New creates and initializes new instance of the rpc test harness. // Optionally, websocket handlers and a specified configuration may be passed. // In the case that a nil config is passed, a default configuration will be -// used. +// used. If a custom btcd executable is specified, it will be used to start the +// harness node. Otherwise a new binary is built on demand. // // NOTE: This function is safe for concurrent access. func New(activeNet *chaincfg.Params, handlers *rpcclient.NotificationHandlers, - extraArgs []string) (*Harness, error) { + extraArgs []string, customExePath string) (*Harness, error) { harnessStateMtx.Lock() defer harnessStateMtx.Unlock() @@ -144,13 +159,15 @@ func New(activeNet *chaincfg.Params, handlers *rpcclient.NotificationHandlers, miningAddr := fmt.Sprintf("--miningaddr=%s", wallet.coinbaseAddr) extraArgs = append(extraArgs, miningAddr) - config, err := newConfig("rpctest", certFile, keyFile, extraArgs) + config, err := newConfig( + "rpctest", certFile, keyFile, extraArgs, customExePath, + ) if err != nil { return nil, err } // Generate p2p+rpc listening addresses. - config.listen, config.rpcListen = generateListeningAddresses() + config.listen, config.rpcListen = ListenAddressGenerator() // Create the testing node bounded to the simnet. node, err := newNode(config, nodeTestData) @@ -190,13 +207,14 @@ func New(activeNet *chaincfg.Params, handlers *rpcclient.NotificationHandlers, } h := &Harness{ - handlers: handlers, - node: node, - maxConnRetries: 20, - testNodeDir: nodeTestData, - ActiveNet: activeNet, - nodeNum: nodeNum, - wallet: wallet, + handlers: handlers, + node: node, + MaxConnRetries: DefaultMaxConnectionRetries, + ConnectionRetryTimeout: DefaultConnectionRetryTimeout, + testNodeDir: nodeTestData, + ActiveNet: activeNet, + nodeNum: nodeNum, + wallet: wallet, } // Track this newly created test instance within the package level @@ -228,13 +246,13 @@ func (h *Harness) SetUp(createTestChain bool, numMatureOutputs uint32) error { // Filter transactions that pay to the coinbase associated with the // wallet. filterAddrs := []btcutil.Address{h.wallet.coinbaseAddr} - if err := h.Node.LoadTxFilter(true, filterAddrs, nil); err != nil { + if err := h.Client.LoadTxFilter(true, filterAddrs, nil); err != nil { return err } // Ensure btcd properly dispatches our registered call-back for each new // block. Otherwise, the memWallet won't function properly. - if err := h.Node.NotifyBlocks(); err != nil { + if err := h.Client.NotifyBlocks(); err != nil { return err } @@ -243,7 +261,7 @@ func (h *Harness) SetUp(createTestChain bool, numMatureOutputs uint32) error { if createTestChain && numMatureOutputs != 0 { numToGenerate := (uint32(h.ActiveNet.CoinbaseMaturity) + numMatureOutputs) - _, err := h.Node.Generate(numToGenerate) + _, err := h.Client.Generate(numToGenerate) if err != nil { return err } @@ -251,7 +269,7 @@ func (h *Harness) SetUp(createTestChain bool, numMatureOutputs uint32) error { // Block until the wallet has fully synced up to the tip of the main // chain. - _, height, err := h.Node.GetBestBlock() + _, height, err := h.Client.GetBestBlock() if err != nil { return err } @@ -272,8 +290,12 @@ func (h *Harness) SetUp(createTestChain bool, numMatureOutputs uint32) error { // // This function MUST be called with the harness state mutex held (for writes). func (h *Harness) tearDown() error { - if h.Node != nil { - h.Node.Shutdown() + if h.Client != nil { + h.Client.Shutdown() + } + + if h.BatchClient != nil { + h.BatchClient.Shutdown() } if err := h.node.shutdown(); err != nil { @@ -308,24 +330,38 @@ func (h *Harness) TearDown() error { // we're not able to establish a connection, this function returns with an // error. func (h *Harness) connectRPCClient() error { - var client *rpcclient.Client + var client, batchClient *rpcclient.Client var err error rpcConf := h.node.config.rpcConnConfig() - for i := 0; i < h.maxConnRetries; i++ { - if client, err = rpcclient.New(&rpcConf, h.handlers); err != nil { - time.Sleep(time.Duration(i) * 50 * time.Millisecond) - continue + batchConf := h.node.config.rpcConnConfig() + batchConf.HTTPPostMode = true + for i := 0; i < h.MaxConnRetries; i++ { + fail := false + if client == nil { + if client, err = rpcclient.New(&rpcConf, h.handlers); err != nil { + time.Sleep(time.Duration(i) * h.ConnectionRetryTimeout) + fail = true + } + } + if batchClient == nil { + if batchClient, err = rpcclient.NewBatch(&batchConf); err != nil { + time.Sleep(time.Duration(i) * h.ConnectionRetryTimeout) + fail = true + } + } + if !fail { + break } - break } - if client == nil { + if client == nil || batchClient == nil { return fmt.Errorf("connection timeout") } - h.Node = client + h.Client = client h.wallet.SetRPCClient(client) + h.BatchClient = batchClient return nil } @@ -447,11 +483,11 @@ func (h *Harness) GenerateAndSubmitBlockWithCustomCoinbaseOutputs( blockVersion = BlockVersion } - prevBlockHash, prevBlockHeight, err := h.Node.GetBestBlock() + prevBlockHash, prevBlockHeight, err := h.Client.GetBestBlock() if err != nil { return nil, err } - mBlock, err := h.Node.GetBlock(prevBlockHash) + mBlock, err := h.Client.GetBlock(prevBlockHash) if err != nil { return nil, err } @@ -466,7 +502,7 @@ func (h *Harness) GenerateAndSubmitBlockWithCustomCoinbaseOutputs( } // Submit the block to the simnet node. - if err := h.Node.SubmitBlock(newBlock, nil); err != nil { + if err := h.Client.SubmitBlock(newBlock, nil); err != nil { return nil, err } @@ -477,13 +513,14 @@ func (h *Harness) GenerateAndSubmitBlockWithCustomCoinbaseOutputs( // addresses designated for the current rpc test. If there haven't been any // test instances created, the default ports are used. Otherwise, in order to // support multiple test nodes running at once, the p2p and rpc port are -// incremented after each initialization. +// picked at random between {min/max}PeerPort and {min/max}RPCPort respectively. func generateListeningAddresses() (string, string) { localhost := "127.0.0.1" + rand.Seed(time.Now().UnixNano()) + portString := func(minPort, maxPort int) string { - port := minPort + numTestInstances + ((20 * processID) % - (maxPort - minPort)) + port := minPort + rand.Intn(maxPort-minPort) return strconv.Itoa(port) } diff --git a/integration/rpctest/rpc_harness_test.go b/integration/rpctest/rpc_harness_test.go index 717f8f45af..df753e3126 100644 --- a/integration/rpctest/rpc_harness_test.go +++ b/integration/rpctest/rpc_harness_test.go @@ -43,7 +43,7 @@ func testSendOutputs(r *Harness, t *testing.T) { } assertTxMined := func(txid *chainhash.Hash, blockHash *chainhash.Hash) { - block, err := r.Node.GetBlock(blockHash) + block, err := r.Client.GetBlock(blockHash) if err != nil { t.Fatalf("unable to get block: %v", err) } @@ -67,7 +67,7 @@ func testSendOutputs(r *Harness, t *testing.T) { // Generate a single block, the transaction the wallet created should // be found in this block. - blockHashes, err := r.Node.Generate(1) + blockHashes, err := r.Client.Generate(1) if err != nil { t.Fatalf("unable to generate single block: %v", err) } @@ -76,7 +76,7 @@ func testSendOutputs(r *Harness, t *testing.T) { // Next, generate a spend much greater than the block reward. This // transaction should also have been mined properly. txid = genSpend(btcutil.Amount(500 * btcutil.SatoshiPerBitcoin)) - blockHashes, err = r.Node.Generate(1) + blockHashes, err = r.Client.Generate(1) if err != nil { t.Fatalf("unable to generate single block: %v", err) } @@ -84,7 +84,7 @@ func testSendOutputs(r *Harness, t *testing.T) { } func assertConnectedTo(t *testing.T, nodeA *Harness, nodeB *Harness) { - nodeAPeers, err := nodeA.Node.GetPeerInfo() + nodeAPeers, err := nodeA.Client.GetPeerInfo() if err != nil { t.Fatalf("unable to get nodeA's peer info") } @@ -105,7 +105,7 @@ func assertConnectedTo(t *testing.T, nodeA *Harness, nodeB *Harness) { func testConnectNode(r *Harness, t *testing.T) { // Create a fresh test harness. - harness, err := New(&chaincfg.SimNetParams, nil, nil) + harness, err := New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatal(err) } @@ -153,7 +153,7 @@ func testActiveHarnesses(r *Harness, t *testing.T) { numInitialHarnesses := len(ActiveHarnesses()) // Create a single test harness. - harness1, err := New(&chaincfg.SimNetParams, nil, nil) + harness1, err := New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatal(err) } @@ -170,7 +170,7 @@ func testActiveHarnesses(r *Harness, t *testing.T) { func testJoinMempools(r *Harness, t *testing.T) { // Assert main test harness has no transactions in its mempool. - pooledHashes, err := r.Node.GetRawMempool() + pooledHashes, err := r.Client.GetRawMempool() if err != nil { t.Fatalf("unable to get mempool for main test harness: %v", err) } @@ -181,7 +181,7 @@ func testJoinMempools(r *Harness, t *testing.T) { // Create a local test harness with only the genesis block. The nodes // will be synced below so the same transaction can be sent to both // nodes without it being an orphan. - harness, err := New(&chaincfg.SimNetParams, nil, nil) + harness, err := New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatal(err) } @@ -210,7 +210,7 @@ func testJoinMempools(r *Harness, t *testing.T) { if err != nil { t.Fatalf("coinbase spend failed: %v", err) } - if _, err := r.Node.SendRawTransaction(testTx, true); err != nil { + if _, err := r.Client.SendRawTransaction(testTx, true); err != nil { t.Fatalf("send transaction failed: %v", err) } @@ -219,7 +219,7 @@ func testJoinMempools(r *Harness, t *testing.T) { harnessSynced := make(chan struct{}) go func() { for { - poolHashes, err := r.Node.GetRawMempool() + poolHashes, err := r.Client.GetRawMempool() if err != nil { t.Fatalf("failed to retrieve harness mempool: %v", err) } @@ -262,7 +262,7 @@ func testJoinMempools(r *Harness, t *testing.T) { // Send the transaction to the local harness which will result in synced // mempools. - if _, err := harness.Node.SendRawTransaction(testTx, true); err != nil { + if _, err := harness.Client.SendRawTransaction(testTx, true); err != nil { t.Fatalf("send transaction failed: %v", err) } @@ -281,7 +281,7 @@ func testJoinMempools(r *Harness, t *testing.T) { func testJoinBlocks(r *Harness, t *testing.T) { // Create a second harness with only the genesis block so it is behind // the main harness. - harness, err := New(&chaincfg.SimNetParams, nil, nil) + harness, err := New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatal(err) } @@ -469,7 +469,7 @@ func testGenerateAndSubmitBlockWithCustomCoinbaseOutputs(r *Harness, func testMemWalletReorg(r *Harness, t *testing.T) { // Create a fresh harness, we'll be using the main harness to force a // re-org on this local harness. - harness, err := New(&chaincfg.SimNetParams, nil, nil) + harness, err := New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { t.Fatal(err) } @@ -566,7 +566,7 @@ const ( func TestMain(m *testing.M) { var err error - mainHarness, err = New(&chaincfg.SimNetParams, nil, nil) + mainHarness, err = New(&chaincfg.SimNetParams, nil, nil, "") if err != nil { fmt.Println("unable to create main harness: ", err) os.Exit(1) @@ -612,7 +612,7 @@ func TestHarness(t *testing.T) { // Current tip should be at a height of numMatureOutputs plus the // required number of blocks for coinbase maturity. - nodeInfo, err := mainHarness.Node.GetInfo() + nodeInfo, err := mainHarness.Client.GetInfo() if err != nil { t.Fatalf("unable to execute getinfo on node: %v", err) } diff --git a/integration/rpctest/utils.go b/integration/rpctest/utils.go index fc7d938dcf..d4d76f2ee6 100644 --- a/integration/rpctest/utils.go +++ b/integration/rpctest/utils.go @@ -49,7 +49,7 @@ func syncMempools(nodes []*Harness) error { retry: for !poolsMatch { - firstPool, err := nodes[0].Node.GetRawMempool() + firstPool, err := nodes[0].Client.GetRawMempool() if err != nil { return err } @@ -58,7 +58,7 @@ retry: // first node, then we're done. Otherwise, drop back to the top // of the loop and retry after a short wait period. for _, node := range nodes[1:] { - nodePool, err := node.Node.GetRawMempool() + nodePool, err := node.Client.GetRawMempool() if err != nil { return err } @@ -84,7 +84,7 @@ retry: var prevHash *chainhash.Hash var prevHeight int32 for _, node := range nodes { - blockHash, blockHeight, err := node.Node.GetBestBlock() + blockHash, blockHeight, err := node.Client.GetBestBlock() if err != nil { return err } @@ -108,24 +108,24 @@ retry: // therefore in the case of disconnects, "from" will attempt to reestablish a // connection to the "to" harness. func ConnectNode(from *Harness, to *Harness) error { - peerInfo, err := from.Node.GetPeerInfo() + peerInfo, err := from.Client.GetPeerInfo() if err != nil { return err } numPeers := len(peerInfo) targetAddr := to.node.config.listen - if err := from.Node.AddNode(targetAddr, rpcclient.ANAdd); err != nil { + if err := from.Client.AddNode(targetAddr, rpcclient.ANAdd); err != nil { return err } // Block until a new connection has been established. - peerInfo, err = from.Node.GetPeerInfo() + peerInfo, err = from.Client.GetPeerInfo() if err != nil { return err } for len(peerInfo) <= numPeers { - peerInfo, err = from.Node.GetPeerInfo() + peerInfo, err = from.Client.GetPeerInfo() if err != nil { return err } diff --git a/mempool/README.md b/mempool/README.md index d9b8d73b5c..5f1e4a4cd1 100644 --- a/mempool/README.md +++ b/mempool/README.md @@ -1,9 +1,9 @@ mempool ======= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/mempool) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/mempool) Package mempool provides a policy-enforced pool of unmined bitcoin transactions. diff --git a/mempool/estimatefee.go b/mempool/estimatefee.go index 3546f6b356..55fe4810df 100644 --- a/mempool/estimatefee.go +++ b/mempool/estimatefee.go @@ -563,7 +563,7 @@ func (ef *FeeEstimator) EstimateFee(numBlocks uint32) (BtcPerKilobyte, error) { if numBlocks > estimateFeeDepth { return -1, fmt.Errorf( "can only estimate fees for up to %d blocks from now", - estimateFeeBinSize) + estimateFeeDepth) } // If there are no cached results, generate them. diff --git a/mempool/mempool_test.go b/mempool/mempool_test.go index 6d43cfd86e..96d5054417 100644 --- a/mempool/mempool_test.go +++ b/mempool/mempool_test.go @@ -1749,6 +1749,47 @@ func TestRBF(t *testing.T) { }, err: "", }, + { + // A transaction that doesn't signal replacement, can + // be replaced if the parent signals replacement. + name: "inherited replacement", + setup: func(ctx *testContext) (*btcutil.Tx, []*btcutil.Tx) { + coinbase := ctx.addCoinbaseTx(1) + + // Create an initial parent transaction that + // marks replacement, we won't be replacing + // this directly however. + coinbaseOut := txOutToSpendableOut(coinbase, 0) + outs := []spendableOutput{coinbaseOut} + parent := ctx.addSignedTx( + outs, 1, defaultFee, true, false, + ) + + // Now create a transaction that spends that + // parent transaction, which is marked as NOT + // being RBF-able. + parentOut := txOutToSpendableOut(parent, 0) + parentOuts := []spendableOutput{parentOut} + childNoReplace := ctx.addSignedTx( + parentOuts, 1, defaultFee, false, false, + ) + + // Now we'll create another transaction that + // replaces the *child* only. This should work + // as the parent has been marked for RBF, even + // though the child hasn't. + respendOuts := []spendableOutput{parentOut} + childReplace, err := ctx.harness.CreateSignedTx( + respendOuts, 1, defaultFee*3, false, + ) + if err != nil { + ctx.t.Fatalf("unable to create child tx: %v", err) + } + + return childReplace, []*btcutil.Tx{childNoReplace} + }, + err: "", + }, } for _, testCase := range testCases { diff --git a/mining/README.md b/mining/README.md index 5295215f8e..3abd195355 100644 --- a/mining/README.md +++ b/mining/README.md @@ -1,9 +1,9 @@ mining ====== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/mining) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/mining) ## Overview diff --git a/mining/cpuminer/README.md b/mining/cpuminer/README.md index 205b8f0360..47247be98f 100644 --- a/mining/cpuminer/README.md +++ b/mining/cpuminer/README.md @@ -1,9 +1,9 @@ cpuminer ======== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/mining/cpuminer) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/mining/cpuminer) ======= ## Overview diff --git a/mining/mining.go b/mining/mining.go index 44ec7dc761..e918328df8 100644 --- a/mining/mining.go +++ b/mining/mining.go @@ -803,40 +803,7 @@ mempoolLoop: // OP_RETURN output within the coinbase transaction. var witnessCommitment []byte if witnessIncluded { - // The witness of the coinbase transaction MUST be exactly 32-bytes - // of all zeroes. - var witnessNonce [blockchain.CoinbaseWitnessDataLen]byte - coinbaseTx.MsgTx().TxIn[0].Witness = wire.TxWitness{witnessNonce[:]} - - // Next, obtain the merkle root of a tree which consists of the - // wtxid of all transactions in the block. The coinbase - // transaction will have a special wtxid of all zeroes. - witnessMerkleTree := blockchain.BuildMerkleTreeStore(blockTxns, - true) - witnessMerkleRoot := witnessMerkleTree[len(witnessMerkleTree)-1] - - // The preimage to the witness commitment is: - // witnessRoot || coinbaseWitness - var witnessPreimage [64]byte - copy(witnessPreimage[:32], witnessMerkleRoot[:]) - copy(witnessPreimage[32:], witnessNonce[:]) - - // The witness commitment itself is the double-sha256 of the - // witness preimage generated above. With the commitment - // generated, the witness script for the output is: OP_RETURN - // OP_DATA_36 {0xaa21a9ed || witnessCommitment}. The leading - // prefix is referred to as the "witness magic bytes". - witnessCommitment = chainhash.DoubleHashB(witnessPreimage[:]) - witnessScript := append(blockchain.WitnessMagicBytes, witnessCommitment...) - - // Finally, create the OP_RETURN carrying witness commitment - // output as an additional output within the coinbase. - commitmentOutput := &wire.TxOut{ - Value: 0, - PkScript: witnessScript, - } - coinbaseTx.MsgTx().TxOut = append(coinbaseTx.MsgTx().TxOut, - commitmentOutput) + witnessCommitment = AddWitnessCommitment(coinbaseTx, blockTxns) } // Calculate the required difficulty for the block. The timestamp @@ -895,6 +862,49 @@ mempoolLoop: }, nil } +// AddWitnessCommitment adds the witness commitment as an OP_RETURN outpout +// within the coinbase tx. The raw commitment is returned. +func AddWitnessCommitment(coinbaseTx *btcutil.Tx, + blockTxns []*btcutil.Tx) []byte { + + // The witness of the coinbase transaction MUST be exactly 32-bytes + // of all zeroes. + var witnessNonce [blockchain.CoinbaseWitnessDataLen]byte + coinbaseTx.MsgTx().TxIn[0].Witness = wire.TxWitness{witnessNonce[:]} + + // Next, obtain the merkle root of a tree which consists of the + // wtxid of all transactions in the block. The coinbase + // transaction will have a special wtxid of all zeroes. + witnessMerkleTree := blockchain.BuildMerkleTreeStore(blockTxns, + true) + witnessMerkleRoot := witnessMerkleTree[len(witnessMerkleTree)-1] + + // The preimage to the witness commitment is: + // witnessRoot || coinbaseWitness + var witnessPreimage [64]byte + copy(witnessPreimage[:32], witnessMerkleRoot[:]) + copy(witnessPreimage[32:], witnessNonce[:]) + + // The witness commitment itself is the double-sha256 of the + // witness preimage generated above. With the commitment + // generated, the witness script for the output is: OP_RETURN + // OP_DATA_36 {0xaa21a9ed || witnessCommitment}. The leading + // prefix is referred to as the "witness magic bytes". + witnessCommitment := chainhash.DoubleHashB(witnessPreimage[:]) + witnessScript := append(blockchain.WitnessMagicBytes, witnessCommitment...) + + // Finally, create the OP_RETURN carrying witness commitment + // output as an additional output within the coinbase. + commitmentOutput := &wire.TxOut{ + Value: 0, + PkScript: witnessScript, + } + coinbaseTx.MsgTx().TxOut = append(coinbaseTx.MsgTx().TxOut, + commitmentOutput) + + return witnessCommitment +} + // UpdateBlockTime updates the timestamp in the header of the passed block to // the current time while taking into account the median time of the last // several blocks to ensure the new time is after that time per the chain diff --git a/netsync/README.md b/netsync/README.md index d5e141151d..a4966815fd 100644 --- a/netsync/README.md +++ b/netsync/README.md @@ -1,9 +1,9 @@ netsync ======= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/netsync) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/netsync) ## Overview diff --git a/netsync/manager.go b/netsync/manager.go index 603fca6ec8..2b6c041156 100644 --- a/netsync/manager.go +++ b/netsync/manager.go @@ -1031,11 +1031,16 @@ func (sm *SyncManager) handleNotFoundMsg(nfmsg *notFoundMsg) { // verify the hash was actually announced by the peer // before deleting from the global requested maps. switch inv.Type { + case wire.InvTypeWitnessBlock: + fallthrough case wire.InvTypeBlock: if _, exists := state.requestedBlocks[inv.Hash]; exists { delete(state.requestedBlocks, inv.Hash) delete(sm.requestedBlocks, inv.Hash) } + + case wire.InvTypeWitnessTx: + fallthrough case wire.InvTypeTx: if _, exists := state.requestedTxns[inv.Hash]; exists { delete(state.requestedTxns, inv.Hash) diff --git a/params.go b/params.go index 14eeff0717..b4d1453dfb 100644 --- a/params.go +++ b/params.go @@ -55,6 +55,13 @@ var simNetParams = params{ rpcPort: "18556", } +// sigNetParams contains parameters specific to the Signet network +// (wire.SigNet). +var sigNetParams = params{ + Params: &chaincfg.SigNetParams, + rpcPort: "38332", +} + // netName returns the name used when referring to a bitcoin network. At the // time of writing, btcd currently places blocks for testnet version 3 in the // data and log directory "testnet", which does not match the Name field of the diff --git a/peer/README.md b/peer/README.md index 1a0ba36e8d..217f5dc3dc 100644 --- a/peer/README.md +++ b/peer/README.md @@ -1,9 +1,9 @@ peer ==== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/peer) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/peer) Package peer provides a common base for creating and managing bitcoin network peers. @@ -63,7 +63,7 @@ $ go get -u github.com/btcsuite/btcd/peer ## Examples -* [New Outbound Peer Example](https://godoc.org/github.com/btcsuite/btcd/peer#example-package--NewOutboundPeer) +* [New Outbound Peer Example](https://pkg.go.dev/github.com/btcsuite/btcd/peer#example-package--NewOutboundPeer) Demonstrates the basic process for initializing and creating an outbound peer. Peers negotiate by exchanging version and verack messages. For demonstration, a simple handler for the version message is attached to the peer. diff --git a/peer/doc.go b/peer/doc.go index cd822fe1cf..88fae8e850 100644 --- a/peer/doc.go +++ b/peer/doc.go @@ -145,6 +145,6 @@ raw message bytes using a format similar to hexdump -C. Bitcoin Improvement Proposals This package supports all BIPS supported by the wire package. -(https://godoc.org/github.com/btcsuite/btcd/wire#hdr-Bitcoin_Improvement_Proposals) +(https://pkg.go.dev/github.com/btcsuite/btcd/wire#hdr-Bitcoin_Improvement_Proposals) */ package peer diff --git a/peer/example_test.go b/peer/example_test.go index cb67683bae..d4662a2b4c 100644 --- a/peer/example_test.go +++ b/peer/example_test.go @@ -25,6 +25,7 @@ func mockRemotePeer() error { UserAgentVersion: "1.0.0", // User agent version to advertise. ChainParams: &chaincfg.SimNetParams, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } // Accept connections on the simnet port. @@ -81,6 +82,7 @@ func Example_newOutboundPeer() { verack <- struct{}{} }, }, + AllowSelfConns: true, } p, err := peer.NewOutboundPeer(peerCfg, "127.0.0.1:18555") if err != nil { diff --git a/peer/export_test.go b/peer/export_test.go deleted file mode 100644 index 06ec78a1a5..0000000000 --- a/peer/export_test.go +++ /dev/null @@ -1,18 +0,0 @@ -// Copyright (c) 2015 The btcsuite developers -// Use of this source code is governed by an ISC -// license that can be found in the LICENSE file. - -/* -This test file is part of the peer package rather than than the peer_test -package so it can bridge access to the internals to properly test cases which -are either not possible or can't reliably be tested via the public interface. -The functions are only exported while the tests are being run. -*/ - -package peer - -// TstAllowSelfConns allows the test package to allow self connections by -// disabling the detection logic. -func TstAllowSelfConns() { - allowSelfConns = true -} diff --git a/peer/peer.go b/peer/peer.go index 82010f3b84..92ac3d27e8 100644 --- a/peer/peer.go +++ b/peer/peer.go @@ -84,11 +84,6 @@ var ( // sentNonces houses the unique nonces that are generated when pushing // version messages that are used to detect self connections. sentNonces = lru.NewCache(50) - - // allowSelfConns is only used to allow the tests to bypass the self - // connection detecting and disconnect logic since they intentionally - // do so for testing purposes. - allowSelfConns bool ) // MessageListeners defines callback function pointers to invoke with message @@ -276,6 +271,11 @@ type Config struct { // TrickleInterval is the duration of the ticker which trickles down the // inventory to a peer. TrickleInterval time.Duration + + // AllowSelfConns is only used to allow the tests to bypass the self + // connection detecting and disconnect logic since they intentionally + // do so for testing purposes. + AllowSelfConns bool } // minUint32 is a helper function to return the minimum of two uint32s. @@ -495,6 +495,10 @@ func (p *Peer) String() string { // This function is safe for concurrent access. func (p *Peer) UpdateLastBlockHeight(newHeight int32) { p.statsMtx.Lock() + if newHeight <= p.lastBlock { + p.statsMtx.Unlock() + return + } log.Tracef("Updating last block height of peer %v from %v to %v", p.addr, p.lastBlock, newHeight) p.lastBlock = newHeight @@ -1892,7 +1896,7 @@ func (p *Peer) readRemoteVersionMsg() error { } // Detect self connections. - if !allowSelfConns && sentNonces.Contains(msg.Nonce) { + if !p.cfg.AllowSelfConns && sentNonces.Contains(msg.Nonce) { return errors.New("disconnecting peer connected to self") } diff --git a/peer/peer_test.go b/peer/peer_test.go index fff0ce3fd8..dd7f36aa3a 100644 --- a/peer/peer_test.go +++ b/peer/peer_test.go @@ -238,6 +238,7 @@ func TestPeerConnection(t *testing.T) { ProtocolVersion: wire.RejectVersion, // Configure with older version Services: 0, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } peer2Cfg := &peer.Config{ Listeners: peer1Cfg.Listeners, @@ -247,6 +248,7 @@ func TestPeerConnection(t *testing.T) { ChainParams: &chaincfg.MainNetParams, Services: wire.SFNodeNetwork | wire.SFNodeWitness, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } wantStats1 := peerStats{ @@ -452,6 +454,7 @@ func TestPeerListeners(t *testing.T) { ChainParams: &chaincfg.MainNetParams, Services: wire.SFNodeBloom, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } inConn, outConn := pipe( &conn{raddr: "10.0.0.1:8333"}, @@ -623,6 +626,7 @@ func TestOutboundPeer(t *testing.T) { ChainParams: &chaincfg.MainNetParams, Services: 0, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } r, w := io.Pipe() @@ -764,6 +768,7 @@ func TestUnsupportedVersionPeer(t *testing.T) { ChainParams: &chaincfg.MainNetParams, Services: 0, TrickleInterval: time.Second * 10, + AllowSelfConns: true, } localNA := wire.NewNetAddressIPPort( @@ -874,6 +879,7 @@ func TestDuplicateVersionMsg(t *testing.T) { UserAgentVersion: "1.0", ChainParams: &chaincfg.MainNetParams, Services: 0, + AllowSelfConns: true, } inConn, outConn := pipe( &conn{laddr: "10.0.0.1:9108", raddr: "10.0.0.2:9108"}, @@ -917,7 +923,69 @@ func TestDuplicateVersionMsg(t *testing.T) { } } -func init() { - // Allow self connection when running the tests. - peer.TstAllowSelfConns() +// TestUpdateLastBlockHeight ensures the last block height is set properly +// during the initial version negotiation and is only allowed to advance to +// higher values via the associated update function. +func TestUpdateLastBlockHeight(t *testing.T) { + // Create a pair of peers that are connected to each other using a fake + // connection and the remote peer starting at height 100. + const remotePeerHeight = 100 + verack := make(chan struct{}) + peerCfg := peer.Config{ + Listeners: peer.MessageListeners{ + OnVerAck: func(p *peer.Peer, msg *wire.MsgVerAck) { + verack <- struct{}{} + }, + }, + UserAgentName: "peer", + UserAgentVersion: "1.0", + ChainParams: &chaincfg.MainNetParams, + Services: 0, + AllowSelfConns: true, + } + remotePeerCfg := peerCfg + remotePeerCfg.NewestBlock = func() (*chainhash.Hash, int32, error) { + return &chainhash.Hash{}, remotePeerHeight, nil + } + inConn, outConn := pipe( + &conn{laddr: "10.0.0.1:9108", raddr: "10.0.0.2:9108"}, + &conn{laddr: "10.0.0.2:9108", raddr: "10.0.0.1:9108"}, + ) + localPeer, err := peer.NewOutboundPeer(&peerCfg, inConn.laddr) + if err != nil { + t.Fatalf("NewOutboundPeer: unexpected err: %v\n", err) + } + localPeer.AssociateConnection(outConn) + inPeer := peer.NewInboundPeer(&remotePeerCfg) + inPeer.AssociateConnection(inConn) + + // Wait for the veracks from the initial protocol version negotiation. + for i := 0; i < 2; i++ { + select { + case <-verack: + case <-time.After(time.Second): + t.Fatal("verack timeout") + } + } + + // Ensure the latest block height starts at the value reported by the remote + // peer via its version message. + if height := localPeer.LastBlock(); height != remotePeerHeight { + t.Fatalf("wrong starting height - got %d, want %d", height, + remotePeerHeight) + } + + // Ensure the latest block height is not allowed to go backwards. + localPeer.UpdateLastBlockHeight(remotePeerHeight - 1) + if height := localPeer.LastBlock(); height != remotePeerHeight { + t.Fatalf("height allowed to go backwards - got %d, want %d", height, + remotePeerHeight) + } + + // Ensure the latest block height is allowed to advance. + localPeer.UpdateLastBlockHeight(remotePeerHeight + 1) + if height := localPeer.LastBlock(); height != remotePeerHeight+1 { + t.Fatalf("height not allowed to advance - got %d, want %d", height, + remotePeerHeight+1) + } } diff --git a/release/README.md b/release/README.md index ea01c7d0bb..f145af4023 100644 --- a/release/README.md +++ b/release/README.md @@ -3,10 +3,7 @@ This package contains the build script that the `btcd` project uses in order to build binaries for each new release. As of `go1.13`, with some new build flags, binaries are now reproducible, allowing developers to build the binary on -distinct machines, and end up with a byte-for-byte identical binary. However, -this wasn't _fully_ solved in `go1.13`, as the build system still includes the -directory the binary is built into the binary itself. As a result, our scripts -utilize a work around needed until `go1.13.2`. +distinct machines, and end up with a byte-for-byte identical binary. Every release should note which Go version was used to build the release, so that version should be used for verifying the release. @@ -62,7 +59,7 @@ index 2195175c71..f65cacef7e 100644 appMinor uint = 20 - appPatch uint = 0 + appPatch uint = 1 - + // appPreRelease MUST only contain characters from semanticAlphabet // per the semantic versioning spec. diff --git a/version.go b/version.go @@ -75,7 +72,7 @@ index 92fd60fdd4..fba55b5a37 100644 appMinor uint = 20 - appPatch uint = 0 + appPatch uint = 1 - + // appPreRelease MUST only contain characters from semanticAlphabet // per the semantic versioning spec. ``` @@ -181,4 +178,4 @@ and `go` (matching the same version used in the release): 8. Extract the archive found in the `btcd-` directory created by the release script and recompute the `SHA256` hash of the release binaries (btcd and btcctl) with `shasum -a 256 `. These should match __exactly__ - as the ones noted above. + as the ones noted above. \ No newline at end of file diff --git a/release/release.sh b/release/release.sh index cc65fb6cf1..de49f64122 100755 --- a/release/release.sh +++ b/release/release.sh @@ -39,7 +39,6 @@ cd $MAINDIR # If BTCDBUILDSYS is set the default list is ignored. Useful to release # for a subset of systems/architectures. SYS=${BTCDBUILDSYS:-" - darwin-386 darwin-amd64 dragonfly-amd64 freebsd-386 diff --git a/rpcadapters.go b/rpcadapters.go index 7d2c3a14d0..ddcdf79bf7 100644 --- a/rpcadapters.go +++ b/rpcadapters.go @@ -223,6 +223,15 @@ func (cm *rpcConnManager) RelayTransactions(txns []*mempool.TxDesc) { cm.server.relayTransactions(txns) } +// NodeAddresses returns an array consisting node addresses which can +// potentially be used to find new nodes in the network. +// +// This function is safe for concurrent access and is part of the +// rpcserverConnManager interface implementation. +func (cm *rpcConnManager) NodeAddresses() []*wire.NetAddress { + return cm.server.addrManager.AddressCache() +} + // rpcSyncMgr provides a block manager for use with the RPC server and // implements the rpcserverSyncManager interface. type rpcSyncMgr struct { diff --git a/rpcclient/README.md b/rpcclient/README.md index 06be0d69e1..08b16f7560 100644 --- a/rpcclient/README.md +++ b/rpcclient/README.md @@ -1,9 +1,9 @@ rpcclient ========= -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/rpcclient) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/rpcclient) rpcclient implements a Websocket-enabled Bitcoin JSON-RPC client package written in [Go](http://golang.org/). It provides a robust and easy to use client for @@ -18,7 +18,7 @@ implement and the API is not stable yet. ## Documentation -* [API Reference](http://godoc.org/github.com/btcsuite/btcd/rpcclient) +* [API Reference](https://pkg.go.dev/github.com/btcsuite/btcd/rpcclient) * [btcd Websockets Example](https://github.com/btcsuite/btcd/tree/master/rpcclient/examples/btcdwebsockets) Connects to a btcd RPC server using TLS-secured websockets, registers for block connected and block disconnected notifications, and gets the current diff --git a/rpcclient/chain.go b/rpcclient/chain.go index 9f0c6c684d..d478da7a00 100644 --- a/rpcclient/chain.go +++ b/rpcclient/chain.go @@ -508,6 +508,44 @@ func (c *Client) GetBlockChainInfo() (*btcjson.GetBlockChainInfoResult, error) { return c.GetBlockChainInfoAsync().Receive() } +// FutureGetBlockFilterResult is a future promise to deliver the result of a +// GetBlockFilterAsync RPC invocation (or an applicable error). +type FutureGetBlockFilterResult chan *response + +// Receive waits for the response promised by the future and returns block filter +// result provided by the server. +func (r FutureGetBlockFilterResult) Receive() (*btcjson.GetBlockFilterResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var blockFilter btcjson.GetBlockFilterResult + err = json.Unmarshal(res, &blockFilter) + if err != nil { + return nil, err + } + + return &blockFilter, nil +} + +// GetBlockFilterAsync returns an instance of a type that can be used to get the +// result of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See GetBlockFilter for the blocking version and more details. +func (c *Client) GetBlockFilterAsync(blockHash chainhash.Hash, filterType *btcjson.FilterTypeName) FutureGetBlockFilterResult { + hash := blockHash.String() + + cmd := btcjson.NewGetBlockFilterCmd(hash, filterType) + return c.sendCmd(cmd) +} + +// GetBlockFilter retrieves a BIP0157 content filter for a particular block. +func (c *Client) GetBlockFilter(blockHash chainhash.Hash, filterType *btcjson.FilterTypeName) (*btcjson.GetBlockFilterResult, error) { + return c.GetBlockFilterAsync(blockHash, filterType).Receive() +} + // FutureGetBlockHashResult is a future promise to deliver the result of a // GetBlockHashAsync RPC invocation (or an applicable error). type FutureGetBlockHashResult chan *response @@ -988,6 +1026,44 @@ func (c *Client) GetTxOut(txHash *chainhash.Hash, index uint32, mempool bool) (* return c.GetTxOutAsync(txHash, index, mempool).Receive() } +// FutureGetTxOutSetInfoResult is a future promise to deliver the result of a +// GetTxOutSetInfoAsync RPC invocation (or an applicable error). +type FutureGetTxOutSetInfoResult chan *response + +// Receive waits for the response promised by the future and returns the +// results of GetTxOutSetInfoAsync RPC invocation. +func (r FutureGetTxOutSetInfoResult) Receive() (*btcjson.GetTxOutSetInfoResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + // Unmarshal result as an gettxoutsetinfo result object. + var txOutSetInfo *btcjson.GetTxOutSetInfoResult + err = json.Unmarshal(res, &txOutSetInfo) + if err != nil { + return nil, err + } + + return txOutSetInfo, nil +} + +// GetTxOutSetInfoAsync returns an instance of a type that can be used to get +// the result of the RPC at some future time by invoking the Receive function on +// the returned instance. +// +// See GetTxOutSetInfo for the blocking version and more details. +func (c *Client) GetTxOutSetInfoAsync() FutureGetTxOutSetInfoResult { + cmd := btcjson.NewGetTxOutSetInfoCmd() + return c.sendCmd(cmd) +} + +// GetTxOutSetInfo returns the statistics about the unspent transaction output +// set. +func (c *Client) GetTxOutSetInfo() (*btcjson.GetTxOutSetInfoResult, error) { + return c.GetTxOutSetInfoAsync().Receive() +} + // FutureRescanBlocksResult is a future promise to deliver the result of a // RescanBlocksAsync RPC invocation (or an applicable error). // @@ -1224,3 +1300,85 @@ func (c *Client) GetBlockStatsAsync(hashOrHeight interface{}, stats *[]string) F func (c *Client) GetBlockStats(hashOrHeight interface{}, stats *[]string) (*btcjson.GetBlockStatsResult, error) { return c.GetBlockStatsAsync(hashOrHeight, stats).Receive() } + +// FutureDeriveAddressesResult is a future promise to deliver the result of an +// DeriveAddressesAsync RPC invocation (or an applicable error). +type FutureDeriveAddressesResult chan *response + +// Receive waits for the response promised by the future and derives one or more addresses +// corresponding to the given output descriptor. +func (r FutureDeriveAddressesResult) Receive() (*btcjson.DeriveAddressesResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var deriveAddressesResult btcjson.DeriveAddressesResult + + err = json.Unmarshal(res, &deriveAddressesResult) + if err != nil { + return nil, err + } + + return &deriveAddressesResult, nil +} + +// DeriveAddressesAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See DeriveAddresses for the blocking version and more details. +func (c *Client) DeriveAddressesAsync(descriptor string, descriptorRange *btcjson.DescriptorRange) FutureDeriveAddressesResult { + cmd := btcjson.NewDeriveAddressesCmd(descriptor, descriptorRange) + return c.sendCmd(cmd) +} + +// DeriveAddresses derives one or more addresses corresponding to an output +// descriptor. If a ranged descriptor is used, the end or the range +// (in [begin,end] notation) to derive must be specified. +func (c *Client) DeriveAddresses(descriptor string, descriptorRange *btcjson.DescriptorRange) (*btcjson.DeriveAddressesResult, error) { + return c.DeriveAddressesAsync(descriptor, descriptorRange).Receive() +} + +// FutureGetDescriptorInfoResult is a future promise to deliver the result of a +// GetDescriptorInfoAsync RPC invocation (or an applicable error). +type FutureGetDescriptorInfoResult chan *response + +// Receive waits for the response promised by the future and returns the analysed +// info of the descriptor. +func (r FutureGetDescriptorInfoResult) Receive() (*btcjson.GetDescriptorInfoResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var descriptorInfo btcjson.GetDescriptorInfoResult + + err = json.Unmarshal(res, &descriptorInfo) + if err != nil { + return nil, err + } + + return &descriptorInfo, nil +} + +// GetDescriptorInfoAsync returns an instance of a type that can be used to get +// the result of the RPC at some future time by invoking the Receive function on +// the returned instance. +// +// See GetDescriptorInfo for the blocking version and more details. +func (c *Client) GetDescriptorInfoAsync(descriptor string) FutureGetDescriptorInfoResult { + cmd := btcjson.NewGetDescriptorInfoCmd(descriptor) + return c.sendCmd(cmd) +} + +// GetDescriptorInfo returns the analysed info of a descriptor string, by invoking the +// getdescriptorinfo RPC. +// +// Use this function to analyse a descriptor string, or compute the checksum +// for a descriptor without one. +// +// See btcjson.GetDescriptorInfoResult for details about the result. +func (c *Client) GetDescriptorInfo(descriptor string) (*btcjson.GetDescriptorInfoResult, error) { + return c.GetDescriptorInfoAsync(descriptor).Receive() +} diff --git a/rpcclient/example_test.go b/rpcclient/example_test.go new file mode 100644 index 0000000000..9ba9adadef --- /dev/null +++ b/rpcclient/example_test.go @@ -0,0 +1,156 @@ +// Copyright (c) 2020 The btcsuite developers +// Use of this source code is governed by an ISC +// license that can be found in the LICENSE file. + +package rpcclient + +import ( + "fmt" + "github.com/btcsuite/btcd/btcjson" +) + +var connCfg = &ConnConfig{ + Host: "localhost:8332", + User: "user", + Pass: "pass", + HTTPPostMode: true, + DisableTLS: true, +} + +func ExampleClient_GetDescriptorInfo() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + descriptorInfo, err := client.GetDescriptorInfo( + "wpkh([d34db33f/84h/0h/0h]0279be667ef9dcbbac55a06295Ce870b07029Bfcdb2dce28d959f2815b16f81798)") + if err != nil { + panic(err) + } + + fmt.Printf("%+v\n", descriptorInfo) + // &{Descriptor:wpkh([d34db33f/84'/0'/0']0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)#n9g43y4k Checksum:qwlqgth7 IsRange:false IsSolvable:true HasPrivateKeys:false} +} + +func ExampleClient_ImportMulti() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + requests := []btcjson.ImportMultiRequest{ + { + Descriptor: btcjson.String( + "pkh([f34db33f/44'/0'/0']xpub6Cc939fyHvfB9pPLWd3bSyyQFvgKbwhidca49jGCM5Hz5ypEPGf9JVXB4NBuUfPgoHnMjN6oNgdC9KRqM11RZtL8QLW6rFKziNwHDYhZ6Kx/0/*)#ed7px9nu"), + Range: &btcjson.DescriptorRange{Value: []int{0, 100}}, + Timestamp: btcjson.TimestampOrNow{Value: 0}, // scan from genesis + WatchOnly: btcjson.Bool(true), + KeyPool: btcjson.Bool(false), + Internal: btcjson.Bool(false), + }, + } + opts := &btcjson.ImportMultiOptions{Rescan: true} + + resp, err := client.ImportMulti(requests, opts) + if err != nil { + panic(err) + } + + fmt.Println(resp[0].Success) + // true +} + +func ExampleClient_DeriveAddresses() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + addrs, err := client.DeriveAddresses( + "pkh([f34db33f/44'/0'/0']xpub6Cc939fyHvfB9pPLWd3bSyyQFvgKbwhidca49jGCM5Hz5ypEPGf9JVXB4NBuUfPgoHnMjN6oNgdC9KRqM11RZtL8QLW6rFKziNwHDYhZ6Kx/0/*)#ed7px9nu", + &btcjson.DescriptorRange{Value: []int{0, 2}}) + if err != nil { + panic(err) + } + + fmt.Printf("%+v\n", addrs) + // &[14NjenDKkGGq1McUgoSkeUHJpW3rrKLbPW 1Pn6i3cvdGhqbdgNjXHfbaYfiuviPiymXj 181x1NbgGYKLeMXkDdXEAqepG75EgU8XtG] +} + +func ExampleClient_GetAddressInfo() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + info, err := client.GetAddressInfo("2NF1FbxtUAsvdU4uW1UC2xkBVatp6cYQuJ3") + if err != nil { + panic(err) + } + + fmt.Println(info.Address) // 2NF1FbxtUAsvdU4uW1UC2xkBVatp6cYQuJ3 + fmt.Println(info.ScriptType.String()) // witness_v0_keyhash + fmt.Println(*info.HDKeyPath) // m/49'/1'/0'/0/4 + fmt.Println(info.Embedded.Address) // tb1q3x2h2kh57wzg7jz00jhwn0ycvqtdk2ane37j27 +} + +func ExampleClient_GetWalletInfo() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + info, err := client.GetWalletInfo() + if err != nil { + panic(err) + } + + fmt.Println(info.WalletVersion) // 169900 + fmt.Println(info.TransactionCount) // 22 + fmt.Println(*info.HDSeedID) // eb44e4e9b864ef17e7ba947da746375b000f5d94 + fmt.Println(info.Scanning.Value) // false +} + +func ExampleClient_GetTxOutSetInfo() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + r, err := client.GetTxOutSetInfo() + if err != nil { + panic(err) + } + + fmt.Println(r.TotalAmount.String()) // 20947654.56996054 BTC + fmt.Println(r.BestBlock.String()) // 000000000000005f94116250e2407310463c0a7cf950f1af9ebe935b1c0687ab + fmt.Println(r.TxOuts) // 24280607 + fmt.Println(r.Transactions) // 9285603 + fmt.Println(r.DiskSize) // 1320871611 +} + +func ExampleClient_CreateWallet() { + client, err := New(connCfg, nil) + if err != nil { + panic(err) + } + defer client.Shutdown() + + r, err := client.CreateWallet( + "mywallet", + WithCreateWalletBlank(), + WithCreateWalletPassphrase("secret"), + ) + if err != nil { + panic(err) + } + + fmt.Println(r.Name) // mywallet +} diff --git a/rpcclient/examples/bitcoincorehttpbulk/README.md b/rpcclient/examples/bitcoincorehttpbulk/README.md new file mode 100644 index 0000000000..ca900b6e79 --- /dev/null +++ b/rpcclient/examples/bitcoincorehttpbulk/README.md @@ -0,0 +1,31 @@ +Bitcoin Core Batch HTTP POST Example +============================== + +This example shows how to use the rpclient package to connect to a Bitcoin Core RPC server using HTTP POST and batch JSON-RPC mode with TLS disabled. + +## Running the Example + +The first step is to use `go get` to download and install the rpcclient package: + +```bash +$ go get github.com/btcsuite/btcd/rpcclient +``` + +Next, modify the `main.go` source to specify the correct RPC username and +password for the RPC server: + +```Go + User: "yourrpcuser", + Pass: "yourrpcpass", +``` + +Finally, navigate to the example's directory and run it with: + +```bash +$ cd $GOPATH/src/github.com/btcsuite/btcd/rpcclient/examples/bitcoincorehttp +$ go run *.go +``` + +## License + +This example is licensed under the [copyfree](http://copyfree.org) ISC License. diff --git a/rpcclient/examples/bitcoincorehttpbulk/main.go b/rpcclient/examples/bitcoincorehttpbulk/main.go new file mode 100644 index 0000000000..3dce058da0 --- /dev/null +++ b/rpcclient/examples/bitcoincorehttpbulk/main.go @@ -0,0 +1,46 @@ +// Copyright (c) 2014-2020 The btcsuite developers +// Use of this source code is governed by an ISC +// license that can be found in the LICENSE file. + +package main + +import ( + "fmt" + "log" + + "github.com/btcsuite/btcd/rpcclient" +) + +func main() { + // Connect to local bitcoin core RPC server using HTTP POST mode. + connCfg := &rpcclient.ConnConfig{ + Host: "localhost:8332", + User: "yourrpcuser", + Pass: "yourrpcpass", + DisableConnectOnNew: true, + HTTPPostMode: true, // Bitcoin core only supports HTTP POST mode + DisableTLS: true, // Bitcoin core does not provide TLS by default + } + batchClient, err := rpcclient.NewBatch(connCfg) + defer batchClient.Shutdown() + + if err != nil { + log.Fatal(err) + } + + // batch mode requires async requests + blockCount := batchClient.GetBlockCountAsync() + block1 := batchClient.GetBlockHashAsync(1) + batchClient.GetBlockHashAsync(2) + batchClient.GetBlockHashAsync(3) + block4 := batchClient.GetBlockHashAsync(4) + difficulty := batchClient.GetDifficultyAsync() + + // sends all queued batch requests + batchClient.Send() + + fmt.Println(blockCount.Receive()) + fmt.Println(block1.Receive()) + fmt.Println(block4.Receive()) + fmt.Println(difficulty.Receive()) +} diff --git a/rpcclient/infrastructure.go b/rpcclient/infrastructure.go index 8609e7c5ad..798f027911 100644 --- a/rpcclient/infrastructure.go +++ b/rpcclient/infrastructure.go @@ -163,6 +163,10 @@ type Client struct { // disconnected indicated whether or not the server is disconnected. disconnected bool + // whether or not to batch requests, false unless changed by Batch() + batch bool + batchList *list.List + // retryCount holds the number of times the client has tried to // reconnect to the RPC server. retryCount int64 @@ -220,8 +224,13 @@ func (c *Client) addRequest(jReq *jsonRequest) error { default: } - element := c.requestList.PushBack(jReq) - c.requestMap[jReq.id] = element + if !c.batch { + element := c.requestList.PushBack(jReq) + c.requestMap[jReq.id] = element + } else { + element := c.batchList.PushBack(jReq) + c.requestMap[jReq.id] = element + } return nil } @@ -289,6 +298,41 @@ func (c *Client) trackRegisteredNtfns(cmd interface{}) { } } +// FutureGetBulkResult waits for the responses promised by the future +// and returns them in a channel +type FutureGetBulkResult chan *response + +// Receive waits for the response promised by the future and returns an map +// of results by request id +func (r FutureGetBulkResult) Receive() (BulkResult, error) { + m := make(BulkResult) + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + var arr []IndividualBulkResult + err = json.Unmarshal(res, &arr) + if err != nil { + return nil, err + } + + for _, results := range arr { + m[results.Id] = results + } + + return m, nil +} + +// IndividualBulkResult represents one result +// from a bulk json rpc api +type IndividualBulkResult struct { + Result interface{} `json:"result"` + Error *btcjson.RPCError `json:"error"` + Id uint64 `json:"id"` +} + +type BulkResult = map[uint64]IndividualBulkResult + // inMessage is the first type that an incoming message is unmarshaled // into. It supports both requests (for notification support) and // responses. The partially-unmarshaled message is a notification if @@ -741,7 +785,12 @@ func (c *Client) handleSendPostMessage(details *sendPostDetails) { // Try to unmarshal the response as a regular JSON-RPC response. var resp rawResponse - err = json.Unmarshal(respBytes, &resp) + var batchResponse json.RawMessage + if c.batch { + err = json.Unmarshal(respBytes, &batchResponse) + } else { + err = json.Unmarshal(respBytes, &resp) + } if err != nil { // When the response itself isn't a valid JSON-RPC response // return an error which includes the HTTP status code and raw @@ -751,8 +800,14 @@ func (c *Client) handleSendPostMessage(details *sendPostDetails) { jReq.responseChan <- &response{err: err} return } - - res, err := resp.result() + var res []byte + if c.batch { + // errors must be dealt with downstream since a whole request cannot + // "error out" other than through the status code error handled above + res, err = batchResponse, nil + } else { + res, err = resp.result() + } jReq.responseChan <- &response{result: res, err: err} } @@ -850,6 +905,9 @@ func (c *Client) sendPost(jReq *jsonRequest) { } httpReq.Close = true httpReq.Header.Set("Content-Type", "application/json") + for key, value := range c.config.ExtraHeaders { + httpReq.Header.Set(key, value) + } // Configure basic access authorization. user, pass, err := c.config.getAuth() @@ -872,7 +930,13 @@ func (c *Client) sendRequest(jReq *jsonRequest) { // POST mode, the command is issued via an HTTP client. Otherwise, // the command is issued via the asynchronous websocket channels. if c.config.HTTPPostMode { - c.sendPost(jReq) + if c.batch { + if err := c.addRequest(jReq); err != nil { + log.Warn(err) + } + } else { + c.sendPost(jReq) + } return } @@ -902,6 +966,10 @@ func (c *Client) sendRequest(jReq *jsonRequest) { // future. It handles both websocket and HTTP POST mode depending on the // configuration of the client. func (c *Client) sendCmd(cmd interface{}) chan *response { + rpcVersion := btcjson.RpcVersion1 + if c.batch { + rpcVersion = btcjson.RpcVersion2 + } // Get the method associated with the command. method, err := btcjson.CmdMethod(cmd) if err != nil { @@ -910,7 +978,7 @@ func (c *Client) sendCmd(cmd interface{}) chan *response { // Marshal the command. id := c.NextID() - marshalledJSON, err := btcjson.MarshalCmd(id, cmd) + marshalledJSON, err := btcjson.MarshalCmd(rpcVersion, id, cmd) if err != nil { return newFutureError(err) } @@ -924,6 +992,7 @@ func (c *Client) sendCmd(cmd interface{}) chan *response { marshalledJSON: marshalledJSON, responseChan: responseChan, } + c.sendRequest(jReq) return responseChan @@ -1161,6 +1230,10 @@ type ConnConfig struct { // flag can be set to true to use basic HTTP POST requests instead. HTTPPostMode bool + // ExtraHeaders specifies the extra headers when perform request. It's + // useful when RPC provider need customized headers. + ExtraHeaders map[string]string + // EnableBCInfoHacks is an option provided to enable compatibility hacks // when connecting to blockchain.info RPC server EnableBCInfoHacks bool @@ -1280,6 +1353,9 @@ func dial(config *ConnConfig) (*websocket.Conn, error) { auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(login)) requestHeader := make(http.Header) requestHeader.Add("Authorization", auth) + for key, value := range config.ExtraHeaders { + requestHeader.Add(key, value) + } // Dial the connection. url := fmt.Sprintf("%s://%s/%s", scheme, config.Host, config.Endpoint) @@ -1347,6 +1423,8 @@ func New(config *ConnConfig, ntfnHandlers *NotificationHandlers) (*Client, error httpClient: httpClient, requestMap: make(map[uint64]*list.Element), requestList: list.New(), + batch: false, + batchList: list.New(), ntfnHandlers: ntfnHandlers, ntfnState: newNotificationState(), sendChan: make(chan []byte, sendBufferSize), @@ -1387,6 +1465,24 @@ func New(config *ConnConfig, ntfnHandlers *NotificationHandlers) (*Client, error return client, nil } +// Batch is a factory that creates a client able to interact with the server using +// JSON-RPC 2.0. The client is capable of accepting an arbitrary number of requests +// and having the server process the all at the same time. It's compatible with both +// btcd and bitcoind +func NewBatch(config *ConnConfig) (*Client, error) { + if !config.HTTPPostMode { + return nil, errors.New("http post mode is required to use batch client") + } + // notification parameter is nil since notifications are not supported in POST mode. + client, err := New(config, nil) + if err != nil { + return nil, err + } + client.batch = true //copy the client with changed batch setting + client.start() + return client, nil +} + // Connect establishes the initial websocket connection. This is necessary when // a client was created after setting the DisableConnectOnNew field of the // Config struct. @@ -1524,3 +1620,69 @@ func (c *Client) BackendVersion() (BackendVersion, error) { return *c.backendVersion, nil } + +func (c *Client) sendAsync() FutureGetBulkResult { + // convert the array of marshalled json requests to a single request we can send + responseChan := make(chan *response, 1) + marshalledRequest := []byte("[") + for iter := c.batchList.Front(); iter != nil; iter = iter.Next() { + request := iter.Value.(*jsonRequest) + marshalledRequest = append(marshalledRequest, request.marshalledJSON...) + marshalledRequest = append(marshalledRequest, []byte(",")...) + } + if len(marshalledRequest) > 0 { + // removes the trailing comma to process the request individually + marshalledRequest = marshalledRequest[:len(marshalledRequest)-1] + } + marshalledRequest = append(marshalledRequest, []byte("]")...) + request := jsonRequest{ + id: c.NextID(), + method: "", + cmd: nil, + marshalledJSON: marshalledRequest, + responseChan: responseChan, + } + c.sendPost(&request) + return responseChan +} + +// Marshall's bulk requests and sends to the server +// creates a response channel to receive the response +func (c *Client) Send() error { + // if batchlist is empty, there's nothing to send + if c.batchList.Len() == 0 { + return nil + } + + // clear batchlist in case of an error + defer func() { + c.batchList = list.New() + }() + + result, err := c.sendAsync().Receive() + + if err != nil { + return err + } + + for iter := c.batchList.Front(); iter != nil; iter = iter.Next() { + var requestError error + request := iter.Value.(*jsonRequest) + individualResult := result[request.id] + fullResult, err := json.Marshal(individualResult.Result) + if err != nil { + return err + } + + if individualResult.Error != nil { + requestError = individualResult.Error + } + + result := response{ + result: fullResult, + err: requestError, + } + request.responseChan <- &result + } + return nil +} diff --git a/rpcclient/mining.go b/rpcclient/mining.go index 8717d3278f..431054e155 100644 --- a/rpcclient/mining.go +++ b/rpcclient/mining.go @@ -461,4 +461,39 @@ func (c *Client) SubmitBlock(block *btcutil.Block, options *btcjson.SubmitBlockO return c.SubmitBlockAsync(block, options).Receive() } -// TODO(davec): Implement GetBlockTemplate +// FutureGetBlockTemplateResponse is a future promise to deliver the result of a +// GetBlockTemplateAsync RPC invocation (or an applicable error). +type FutureGetBlockTemplateResponse chan *response + +// Receive waits for the response promised by the future and returns an error if +// any occurred when retrieving the block template. +func (r FutureGetBlockTemplateResponse) Receive() (*btcjson.GetBlockTemplateResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + // Unmarshal result as a getwork result object. + var result btcjson.GetBlockTemplateResult + err = json.Unmarshal(res, &result) + if err != nil { + return nil, err + } + + return &result, nil +} + +// GetBlockTemplateAsync returns an instance of a type that can be used to get the +// result of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See GetBlockTemplate for the blocking version and more details. +func (c *Client) GetBlockTemplateAsync(req *btcjson.TemplateRequest) FutureGetBlockTemplateResponse { + cmd := btcjson.NewGetBlockTemplateCmd(req) + return c.sendCmd(cmd) +} + +// GetBlockTemplate returns a new block template for mining. +func (c *Client) GetBlockTemplate(req *btcjson.TemplateRequest) (*btcjson.GetBlockTemplateResult, error) { + return c.GetBlockTemplateAsync(req).Receive() +} diff --git a/rpcclient/net.go b/rpcclient/net.go index 6eb7362541..8c191ff66f 100644 --- a/rpcclient/net.go +++ b/rpcclient/net.go @@ -90,7 +90,7 @@ func (c *Client) NodeAsync(command btcjson.NodeSubCmd, host string, // connect or diconnect a non-persistent one. // // The connectSubCmd should be set either "perm" or "temp", depending on -// whether we are targetting a persistent or non-persistent peer. Passing nil +// whether we are targeting a persistent or non-persistent peer. Passing nil // will cause the default value to be used, which currently is "temp". func (c *Client) Node(command btcjson.NodeSubCmd, host string, connectSubCmd *string) error { @@ -281,6 +281,43 @@ func (c *Client) GetNetworkInfo() (*btcjson.GetNetworkInfoResult, error) { return c.GetNetworkInfoAsync().Receive() } +// FutureGetNodeAddressesResult is a future promise to deliver the result of a +// GetNodeAddressesAsync RPC invocation (or an applicable error). +type FutureGetNodeAddressesResult chan *response + +// Receive waits for the response promised by the future and returns data about +// known node addresses. +func (r FutureGetNodeAddressesResult) Receive() ([]btcjson.GetNodeAddressesResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + // Unmarshal result as an array of getnodeaddresses result objects. + var nodeAddresses []btcjson.GetNodeAddressesResult + err = json.Unmarshal(res, &nodeAddresses) + if err != nil { + return nil, err + } + + return nodeAddresses, nil +} + +// GetNodeAddressesAsync returns an instance of a type that can be used to get the +// result of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See GetNodeAddresses for the blocking version and more details. +func (c *Client) GetNodeAddressesAsync(count *int32) FutureGetNodeAddressesResult { + cmd := btcjson.NewGetNodeAddressesCmd(count) + return c.sendCmd(cmd) +} + +// GetNodeAddresses returns data about known node addresses. +func (c *Client) GetNodeAddresses(count *int32) ([]btcjson.GetNodeAddressesResult, error) { + return c.GetNodeAddressesAsync(count).Receive() +} + // FutureGetPeerInfoResult is a future promise to deliver the result of a // GetPeerInfoAsync RPC invocation (or an applicable error). type FutureGetPeerInfoResult chan *response diff --git a/rpcclient/rawrequest.go b/rpcclient/rawrequest.go index dd16fd049e..baead8bb90 100644 --- a/rpcclient/rawrequest.go +++ b/rpcclient/rawrequest.go @@ -44,7 +44,7 @@ func (c *Client) RawRequestAsync(method string, params []json.RawMessage) Future // than custom commands. id := c.NextID() rawRequest := &btcjson.Request{ - Jsonrpc: "1.0", + Jsonrpc: btcjson.RpcVersion1, ID: id, Method: method, Params: params, diff --git a/rpcclient/rawtransactions.go b/rpcclient/rawtransactions.go index 4e8d4e4d9c..35038ed9d3 100644 --- a/rpcclient/rawtransactions.go +++ b/rpcclient/rawtransactions.go @@ -555,7 +555,7 @@ func (c *Client) SignRawTransaction4Async(tx *wire.MsgTx, return c.sendCmd(cmd) } -// SignRawTransaction4 signs inputs for the passed transaction using the +// SignRawTransaction4 signs inputs for the passed transaction using // the specified signature hash type given the list of information about extra // input transactions and a potential list of private keys needed to perform // the signing process. The private keys, if specified, must be in wallet @@ -582,6 +582,149 @@ func (c *Client) SignRawTransaction4(tx *wire.MsgTx, hashType).Receive() } +// FutureSignRawTransactionWithWalletResult is a future promise to deliver +// the result of the SignRawTransactionWithWalletAsync RPC invocation (or +// an applicable error). +type FutureSignRawTransactionWithWalletResult chan *response + +// Receive waits for the response promised by the future and returns the +// signed transaction as well as whether or not all inputs are now signed. +func (r FutureSignRawTransactionWithWalletResult) Receive() (*wire.MsgTx, bool, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, false, err + } + + // Unmarshal as a signtransactionwithwallet result. + var signRawTxWithWalletResult btcjson.SignRawTransactionWithWalletResult + err = json.Unmarshal(res, &signRawTxWithWalletResult) + if err != nil { + return nil, false, err + } + + // Decode the serialized transaction hex to raw bytes. + serializedTx, err := hex.DecodeString(signRawTxWithWalletResult.Hex) + if err != nil { + return nil, false, err + } + + // Deserialize the transaction and return it. + var msgTx wire.MsgTx + if err := msgTx.Deserialize(bytes.NewReader(serializedTx)); err != nil { + return nil, false, err + } + + return &msgTx, signRawTxWithWalletResult.Complete, nil +} + +// SignRawTransactionWithWalletAsync returns an instance of a type that can be used +// to get the result of the RPC at some future time by invoking the Receive function +// on the returned instance. +// +// See SignRawTransactionWithWallet for the blocking version and more details. +func (c *Client) SignRawTransactionWithWalletAsync(tx *wire.MsgTx) FutureSignRawTransactionWithWalletResult { + txHex := "" + if tx != nil { + // Serialize the transaction and convert to hex string. + buf := bytes.NewBuffer(make([]byte, 0, tx.SerializeSize())) + if err := tx.Serialize(buf); err != nil { + return newFutureError(err) + } + txHex = hex.EncodeToString(buf.Bytes()) + } + + cmd := btcjson.NewSignRawTransactionWithWalletCmd(txHex, nil, nil) + return c.sendCmd(cmd) +} + +// SignRawTransactionWithWallet signs inputs for the passed transaction and returns +// the signed transaction as well as whether or not all inputs are now signed. +// +// This function assumes the RPC server already knows the input transactions for the +// passed transaction which needs to be signed and uses the default signature hash +// type. Use one of the SignRawTransactionWithWallet# variants to specify that +// information if needed. +func (c *Client) SignRawTransactionWithWallet(tx *wire.MsgTx) (*wire.MsgTx, bool, error) { + return c.SignRawTransactionWithWalletAsync(tx).Receive() +} + +// SignRawTransactionWithWallet2Async returns an instance of a type that can be +// used to get the result of the RPC at some future time by invoking the Receive +// function on the returned instance. +// +// See SignRawTransactionWithWallet2 for the blocking version and more details. +func (c *Client) SignRawTransactionWithWallet2Async(tx *wire.MsgTx, + inputs []btcjson.RawTxWitnessInput) FutureSignRawTransactionWithWalletResult { + + txHex := "" + if tx != nil { + // Serialize the transaction and convert to hex string. + buf := bytes.NewBuffer(make([]byte, 0, tx.SerializeSize())) + if err := tx.Serialize(buf); err != nil { + return newFutureError(err) + } + txHex = hex.EncodeToString(buf.Bytes()) + } + + cmd := btcjson.NewSignRawTransactionWithWalletCmd(txHex, &inputs, nil) + return c.sendCmd(cmd) +} + +// SignRawTransactionWithWallet2 signs inputs for the passed transaction given the +// list of information about the input transactions needed to perform the signing +// process. +// +// This only input transactions that need to be specified are ones the +// RPC server does not already know. Already known input transactions will be +// merged with the specified transactions. +// +// See SignRawTransactionWithWallet if the RPC server already knows the input +// transactions. +func (c *Client) SignRawTransactionWithWallet2(tx *wire.MsgTx, + inputs []btcjson.RawTxWitnessInput) (*wire.MsgTx, bool, error) { + + return c.SignRawTransactionWithWallet2Async(tx, inputs).Receive() +} + +// SignRawTransactionWithWallet3Async returns an instance of a type that can +// be used to get the result of the RPC at some future time by invoking the +// Receive function on the returned instance. +// +// See SignRawTransactionWithWallet3 for the blocking version and more details. +func (c *Client) SignRawTransactionWithWallet3Async(tx *wire.MsgTx, + inputs []btcjson.RawTxWitnessInput, hashType SigHashType) FutureSignRawTransactionWithWalletResult { + + txHex := "" + if tx != nil { + // Serialize the transaction and convert to hex string. + buf := bytes.NewBuffer(make([]byte, 0, tx.SerializeSize())) + if err := tx.Serialize(buf); err != nil { + return newFutureError(err) + } + txHex = hex.EncodeToString(buf.Bytes()) + } + + cmd := btcjson.NewSignRawTransactionWithWalletCmd(txHex, &inputs, btcjson.String(string(hashType))) + return c.sendCmd(cmd) +} + +// SignRawTransactionWithWallet3 signs inputs for the passed transaction using +// the specified signature hash type given the list of information about extra +// input transactions. +// +// The only input transactions that need to be specified are ones the RPC server +// does not already know. This means the list of transaction inputs can be nil +// if the RPC server already knows them all. +// +// This function should only used if a non-default signature hash type is +// desired. Otherwise, see SignRawTransactionWithWallet if the RPC server already +// knows the input transactions, or SignRawTransactionWihWallet2 if it does not. +func (c *Client) SignRawTransactionWithWallet3(tx *wire.MsgTx, + inputs []btcjson.RawTxWitnessInput, hashType SigHashType) (*wire.MsgTx, bool, error) { + + return c.SignRawTransactionWithWallet3Async(tx, inputs, hashType).Receive() +} + // FutureSearchRawTransactionsResult is a future promise to deliver the result // of the SearchRawTransactionsAsync RPC invocation (or an applicable error). type FutureSearchRawTransactionsResult chan *response diff --git a/rpcclient/wallet.go b/rpcclient/wallet.go index 37bf9471e0..011e91062b 100644 --- a/rpcclient/wallet.go +++ b/rpcclient/wallet.go @@ -1,4 +1,4 @@ -// Copyright (c) 2014-2017 The btcsuite developers +// Copyright (c) 2014-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. @@ -164,6 +164,25 @@ func (c *Client) ListTransactionsCountFrom(account string, count, from int) ([]b return c.ListTransactionsCountFromAsync(account, count, from).Receive() } +// ListTransactionsCountFromWatchOnlyAsync returns an instance of a type that can be used +// to get the result of the RPC at some future time by invoking the Receive +// function on the returned instance. +// +// See ListTransactionsCountFromWatchOnly for the blocking version and more details. +func (c *Client) ListTransactionsCountFromWatchOnlyAsync(account string, count, from int, watchOnly bool) FutureListTransactionsResult { + cmd := btcjson.NewListTransactionsCmd(&account, &count, &from, &watchOnly) + return c.sendCmd(cmd) +} + +// ListTransactionsCountFromWatchOnly returns a list of the most recent transactions up +// to the passed count while skipping the first 'from' transactions. It will include or +// exclude transactions from watch-only addresses based on the passed value for the watchOnly parameter +// +// See the ListTransactions and ListTransactionsCount functions to use defaults. +func (c *Client) ListTransactionsCountFromWatchOnly(account string, count, from int, watchOnly bool) ([]btcjson.ListTransactionsResult, error) { + return c.ListTransactionsCountFromWatchOnlyAsync(account, count, from, watchOnly).Receive() +} + // FutureListUnspentResult is a future promise to deliver the result of a // ListUnspentAsync, ListUnspentMinAsync, ListUnspentMinMaxAsync, or // ListUnspentMinMaxAddressesAsync RPC invocation (or an applicable error). @@ -243,8 +262,8 @@ func (c *Client) ListUnspent() ([]btcjson.ListUnspentResult, error) { } // ListUnspentMin returns all unspent transaction outputs known to a wallet, -// using the specified number of minimum conformations and default number of -// maximum confiramtions (999999) as a filter. +// using the specified number of minimum confirmations and default number of +// maximum confirmations (999999) as a filter. func (c *Client) ListUnspentMin(minConf int) ([]btcjson.ListUnspentResult, error) { return c.ListUnspentMinAsync(minConf).Receive() } @@ -307,6 +326,7 @@ func (c *Client) ListSinceBlockAsync(blockHash *chainhash.Hash) FutureListSinceB // minimum confirmations as a filter. // // See ListSinceBlockMinConf to override the minimum number of confirmations. +// See ListSinceBlockMinConfWatchOnly to override the minimum number of confirmations and watch only parameter. func (c *Client) ListSinceBlock(blockHash *chainhash.Hash) (*btcjson.ListSinceBlockResult, error) { return c.ListSinceBlockAsync(blockHash).Receive() } @@ -335,6 +355,30 @@ func (c *Client) ListSinceBlockMinConf(blockHash *chainhash.Hash, minConfirms in return c.ListSinceBlockMinConfAsync(blockHash, minConfirms).Receive() } +// ListSinceBlockMinConfWatchOnlyAsync returns an instance of a type that can be used to +// get the result of the RPC at some future time by invoking the Receive +// function on the returned instance. +// +// See ListSinceBlockMinConfWatchOnly for the blocking version and more details. +func (c *Client) ListSinceBlockMinConfWatchOnlyAsync(blockHash *chainhash.Hash, minConfirms int, watchOnly bool) FutureListSinceBlockResult { + var hash *string + if blockHash != nil { + hash = btcjson.String(blockHash.String()) + } + + cmd := btcjson.NewListSinceBlockCmd(hash, &minConfirms, &watchOnly) + return c.sendCmd(cmd) +} + +// ListSinceBlockMinConfWatchOnly returns all transactions added in blocks since the +// specified block hash, or all transactions if it is nil, using the specified +// number of minimum confirmations as a filter. +// +// See ListSinceBlock to use the default minimum number of confirmations and default watch only parameter. +func (c *Client) ListSinceBlockMinConfWatchOnly(blockHash *chainhash.Hash, minConfirms int, watchOnly bool) (*btcjson.ListSinceBlockResult, error) { + return c.ListSinceBlockMinConfWatchOnlyAsync(blockHash, minConfirms, watchOnly).Receive() +} + // ************************** // Transaction Send Functions // ************************** @@ -528,7 +572,7 @@ func (c *Client) SendToAddressCommentAsync(address btcutil.Address, // SendToAddressComment sends the passed amount to the given address and stores // the provided comment and comment to in the wallet. The comment parameter is // intended to be used for the purpose of the transaction while the commentTo -// parameter is indended to be used for who the transaction is being sent to. +// parameter is intended to be used for who the transaction is being sent to. // // The comments are not part of the transaction and are only internal // to the wallet. @@ -634,7 +678,7 @@ func (c *Client) SendFromCommentAsync(fromAccount string, // SendFromComment sends the passed amount to the given address using the // provided account as a source of funds and stores the provided comment and // comment to in the wallet. The comment parameter is intended to be used for -// the purpose of the transaction while the commentTo parameter is indended to +// the purpose of the transaction while the commentTo parameter is intended to // be used for who the transaction is being sent to. Only funds with the passed // number of minimum confirmations will be used. // @@ -895,6 +939,126 @@ func (c *Client) CreateNewAccount(account string) error { return c.CreateNewAccountAsync(account).Receive() } +// FutureCreateWalletResult is a future promise to deliver the result of a +// CreateWalletAsync RPC invocation (or an applicable error). +type FutureCreateWalletResult chan *response + +// Receive waits for the response promised by the future and returns the +// result of creating a new wallet. +func (r FutureCreateWalletResult) Receive() (*btcjson.CreateWalletResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var createWalletResult btcjson.CreateWalletResult + err = json.Unmarshal(res, &createWalletResult) + if err != nil { + return nil, err + } + return &createWalletResult, nil +} + +// CreateWalletOpt defines a functional-option to be used with CreateWallet +// method. +type CreateWalletOpt func(*btcjson.CreateWalletCmd) + +// WithCreateWalletDisablePrivateKeys disables the possibility of private keys +// to be used with a wallet created using the CreateWallet method. Using this +// option will make the wallet watch-only. +func WithCreateWalletDisablePrivateKeys() CreateWalletOpt { + return func(c *btcjson.CreateWalletCmd) { + c.DisablePrivateKeys = btcjson.Bool(true) + } +} + +// WithCreateWalletBlank specifies creation of a blank wallet. +func WithCreateWalletBlank() CreateWalletOpt { + return func(c *btcjson.CreateWalletCmd) { + c.Blank = btcjson.Bool(true) + } +} + +// WithCreateWalletPassphrase specifies a passphrase to encrypt the wallet +// with. +func WithCreateWalletPassphrase(value string) CreateWalletOpt { + return func(c *btcjson.CreateWalletCmd) { + c.Passphrase = btcjson.String(value) + } +} + +// WithCreateWalletAvoidReuse specifies keeping track of coin reuse, and +// treat dirty and clean coins differently with privacy considerations in mind. +func WithCreateWalletAvoidReuse() CreateWalletOpt { + return func(c *btcjson.CreateWalletCmd) { + c.AvoidReuse = btcjson.Bool(true) + } +} + +// CreateWalletAsync returns an instance of a type that can be used to get the +// result of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See CreateWallet for the blocking version and more details. +func (c *Client) CreateWalletAsync(name string, opts ...CreateWalletOpt) FutureCreateWalletResult { + cmd := btcjson.NewCreateWalletCmd(name, nil, nil, nil, nil) + + // Apply each specified option to mutate the default command. + for _, opt := range opts { + opt(cmd) + } + + return c.sendCmd(cmd) +} + +// CreateWallet creates a new wallet account, with the possibility to use +// private keys. +// +// Optional parameters can be specified using functional-options pattern. The +// following functions are available: +// * WithCreateWalletDisablePrivateKeys +// * WithCreateWalletBlank +// * WithCreateWalletPassphrase +// * WithCreateWalletAvoidReuse +func (c *Client) CreateWallet(name string, opts ...CreateWalletOpt) (*btcjson.CreateWalletResult, error) { + return c.CreateWalletAsync(name, opts...).Receive() +} + +// FutureGetAddressInfoResult is a future promise to deliver the result of an +// GetAddressInfoAsync RPC invocation (or an applicable error). +type FutureGetAddressInfoResult chan *response + +// Receive waits for the response promised by the future and returns the information +// about the given bitcoin address. +func (r FutureGetAddressInfoResult) Receive() (*btcjson.GetAddressInfoResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var getAddressInfoResult btcjson.GetAddressInfoResult + err = json.Unmarshal(res, &getAddressInfoResult) + if err != nil { + return nil, err + } + return &getAddressInfoResult, nil +} + +// GetAddressInfoAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See GetAddressInfo for the blocking version and more details. +func (c *Client) GetAddressInfoAsync(address string) FutureGetAddressInfoResult { + cmd := btcjson.NewGetAddressInfoCmd(address) + return c.sendCmd(cmd) +} + +// GetAddressInfo returns information about the given bitcoin address. +func (c *Client) GetAddressInfo(address string) (*btcjson.GetAddressInfoResult, error) { + return c.GetAddressInfoAsync(address).Receive() +} + // FutureGetNewAddressResult is a future promise to deliver the result of a // GetNewAddressAsync RPC invocation (or an applicable error). type FutureGetNewAddressResult struct { @@ -2203,6 +2367,44 @@ func (c *Client) ImportAddressRescan(address string, account string, rescan bool return c.ImportAddressRescanAsync(address, account, rescan).Receive() } +// FutureImportMultiResult is a future promise to deliver the result of an +// ImportMultiAsync RPC invocation (or an applicable error). +type FutureImportMultiResult chan *response + +// Receive waits for the response promised by the future and returns the result +// of importing multiple addresses/scripts. +func (r FutureImportMultiResult) Receive() (btcjson.ImportMultiResults, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var importMultiResults btcjson.ImportMultiResults + err = json.Unmarshal(res, &importMultiResults) + if err != nil { + return nil, err + } + return importMultiResults, nil +} + +// ImportMultiAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See ImportMulti for the blocking version and more details. +func (c *Client) ImportMultiAsync(requests []btcjson.ImportMultiRequest, options *btcjson.ImportMultiOptions) FutureImportMultiResult { + cmd := btcjson.NewImportMultiCmd(requests, options) + return c.sendCmd(cmd) +} + +// ImportMulti imports addresses/scripts, optionally rescanning the blockchain +// from the earliest creation time of the imported scripts. +// +// See btcjson.ImportMultiRequest for details on the requests parameter. +func (c *Client) ImportMulti(requests []btcjson.ImportMultiRequest, options *btcjson.ImportMultiOptions) (btcjson.ImportMultiResults, error) { + return c.ImportMultiAsync(requests, options).Receive() +} + // FutureImportPrivKeyResult is a future promise to deliver the result of an // ImportPrivKeyAsync RPC invocation (or an applicable error). type FutureImportPrivKeyResult chan *response @@ -2367,13 +2569,265 @@ func (c *Client) GetInfo() (*btcjson.InfoWalletResult, error) { return c.GetInfoAsync().Receive() } +// FutureImportPubKeyResult is a future promise to deliver the result of an +// WalletCreateFundedPsbt RPC invocation (or an applicable error). +type FutureWalletCreateFundedPsbtResult chan *response + +// Receive waits for the response promised by the future and returns the +// partially signed transaction in PSBT format along with the resulting fee +// and change output index. +func (r FutureWalletCreateFundedPsbtResult) Receive() (*btcjson.WalletCreateFundedPsbtResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + // Unmarshal result as a getinfo result object. + var psbtRes btcjson.WalletCreateFundedPsbtResult + err = json.Unmarshal(res, &psbtRes) + if err != nil { + return nil, err + } + + return &psbtRes, nil +} + +// WalletCreateFundedPsbtAsync returns an instance of a type that can be used +// to get the result of the RPC at some future time by invoking the Receive +// function on the returned instance. +// +// See WalletCreateFundedPsbt for the blocking version and more details. +func (c *Client) WalletCreateFundedPsbtAsync( + inputs []btcjson.PsbtInput, outputs []btcjson.PsbtOutput, locktime *uint32, + options *btcjson.WalletCreateFundedPsbtOpts, bip32Derivs *bool, +) FutureWalletCreateFundedPsbtResult { + cmd := btcjson.NewWalletCreateFundedPsbtCmd(inputs, outputs, locktime, options, bip32Derivs) + return c.sendCmd(cmd) +} + +// WalletCreateFundedPsbt creates and funds a transaction in the Partially +// Signed Transaction format. Inputs will be added if supplied inputs are not +// enough. +func (c *Client) WalletCreateFundedPsbt( + inputs []btcjson.PsbtInput, outputs []btcjson.PsbtOutput, locktime *uint32, + options *btcjson.WalletCreateFundedPsbtOpts, bip32Derivs *bool, +) (*btcjson.WalletCreateFundedPsbtResult, error) { + return c.WalletCreateFundedPsbtAsync(inputs, outputs, locktime, options, bip32Derivs).Receive() +} + +// FutureWalletProcessPsbtResult is a future promise to deliver the result of a +// WalletCreateFundedPsb RPC invocation (or an applicable error). +type FutureWalletProcessPsbtResult chan *response + +// Receive waits for the response promised by the future and returns an updated +// PSBT with signed inputs from the wallet and a boolen indicating if the the +// transaction has a complete set of signatures. +func (r FutureWalletProcessPsbtResult) Receive() (*btcjson.WalletProcessPsbtResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + // Unmarshal result as a getinfo result object. + var psbtRes btcjson.WalletProcessPsbtResult + err = json.Unmarshal(res, &psbtRes) + if err != nil { + return nil, err + } + + return &psbtRes, nil +} + +// WalletProcessPsbtAsync returns an instance of a type that can be used +// to get the result of the RPC at some future time by invoking the Receive +// function on the returned instance. +// +// See WalletProcessPsbt for the blocking version and more details. +func (c *Client) WalletProcessPsbtAsync( + psbt string, sign *bool, sighashType SigHashType, bip32Derivs *bool, +) FutureWalletProcessPsbtResult { + cmd := btcjson.NewWalletProcessPsbtCmd(psbt, sign, btcjson.String(sighashType.String()), bip32Derivs) + return c.sendCmd(cmd) +} + +// WalletProcessPsbt updates a PSBT with input information from our wallet and +// then signs inputs. +func (c *Client) WalletProcessPsbt( + psbt string, sign *bool, sighashType SigHashType, bip32Derivs *bool, +) (*btcjson.WalletProcessPsbtResult, error) { + return c.WalletProcessPsbtAsync(psbt, sign, sighashType, bip32Derivs).Receive() +} + +// FutureGetWalletInfoResult is a future promise to deliver the result of an +// GetWalletInfoAsync RPC invocation (or an applicable error). +type FutureGetWalletInfoResult chan *response + +// Receive waits for the response promised by the future and returns the result +// of wallet state info. +func (r FutureGetWalletInfoResult) Receive() (*btcjson.GetWalletInfoResult, error) { + res, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var getWalletInfoResult btcjson.GetWalletInfoResult + err = json.Unmarshal(res, &getWalletInfoResult) + if err != nil { + return nil, err + } + return &getWalletInfoResult, nil +} + +// GetWalletInfoAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See GetWalletInfo for the blocking version and more details. +func (c *Client) GetWalletInfoAsync() FutureGetWalletInfoResult { + cmd := btcjson.NewGetWalletInfoCmd() + return c.sendCmd(cmd) +} + +// GetWalletInfo returns various wallet state info. +func (c *Client) GetWalletInfo() (*btcjson.GetWalletInfoResult, error) { + return c.GetWalletInfoAsync().Receive() +} + +// FutureBackupWalletResult is a future promise to deliver the result of an +// BackupWalletAsync RPC invocation (or an applicable error) +type FutureBackupWalletResult chan *response + +// Receive waits for the response promised by the future +func (r FutureBackupWalletResult) Receive() error { + _, err := receiveFuture(r) + return err +} + +// BackupWalletAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See BackupWallet for the blocking version and more details. +func (c *Client) BackupWalletAsync(destination string) FutureBackupWalletResult { + return c.sendCmd(btcjson.NewBackupWalletCmd(destination)) +} + +// BackupWallet safely copies current wallet file to destination, which can +// be a directory or a path with filename +func (c *Client) BackupWallet(destination string) error { + return c.BackupWalletAsync(destination).Receive() +} + +// FutureDumpWalletResult is a future promise to deliver the result of an +// DumpWallet RPC invocation (or an applicable error) +type FutureDumpWalletResult chan *response + +// Receive waits for the response promised by the future +func (r FutureDumpWalletResult) Receive() (*btcjson.DumpWalletResult, error) { + bytes, err := receiveFuture(r) + if err != nil { + return nil, err + } + + var res btcjson.DumpWalletResult + err = json.Unmarshal(bytes, &res) + return &res, err +} + +// DumpWalletAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See DumpWalletAsync for the blocking version and more details. +func (c *Client) DumpWalletAsync(destination string) FutureDumpWalletResult { + return c.sendCmd(btcjson.NewDumpWalletCmd(destination)) +} + +// DumpWallet dumps all wallet keys in a human-readable format to a server-side file. +func (c *Client) DumpWallet(destination string) (*btcjson.DumpWalletResult, error) { + return c.DumpWalletAsync(destination).Receive() +} + +// FutureImportWalletResult is a future promise to deliver the result of an +// ImportWalletAsync RPC invocation (or an applicable error) +type FutureImportWalletResult chan *response + +// Receive waits for the response promised by the future +func (r FutureImportWalletResult) Receive() error { + _, err := receiveFuture(r) + return err +} + +// ImportWalletAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See ImportWallet for the blocking version and more details. +func (c *Client) ImportWalletAsync(filename string) FutureImportWalletResult { + return c.sendCmd(btcjson.NewImportWalletCmd(filename)) +} + +// ImportWallet imports keys from a wallet dump file (see DumpWallet). +func (c *Client) ImportWallet(filename string) error { + return c.ImportWalletAsync(filename).Receive() +} + +// FutureUnloadWalletResult is a future promise to deliver the result of an +// UnloadWalletAsync RPC invocation (or an applicable error) +type FutureUnloadWalletResult chan *response + +// Receive waits for the response promised by the future +func (r FutureUnloadWalletResult) Receive() error { + _, err := receiveFuture(r) + return err +} + +// UnloadWalletAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See UnloadWallet for the blocking version and more details. +func (c *Client) UnloadWalletAsync(walletName *string) FutureUnloadWalletResult { + return c.sendCmd(btcjson.NewUnloadWalletCmd(walletName)) +} + +// UnloadWallet unloads the referenced wallet. If the RPC server URL already +// contains the name of the wallet, like http://127.0.0.1:8332/wallet/, +// the parameter must be nil, or it'll return an error. +func (c *Client) UnloadWallet(walletName *string) error { + return c.UnloadWalletAsync(walletName).Receive() +} + +// FutureLoadWalletResult is a future promise to deliver the result of an +// LoadWalletAsync RPC invocation (or an applicable error) +type FutureLoadWalletResult chan *response + +// Receive waits for the response promised by the future +func (r FutureLoadWalletResult) Receive() (*btcjson.LoadWalletResult, error) { + bytes, err := receiveFuture(r) + if err != nil { + return nil, err + } + var result btcjson.LoadWalletResult + err = json.Unmarshal(bytes, &result) + return &result, err +} + +// LoadWalletAsync returns an instance of a type that can be used to get the result +// of the RPC at some future time by invoking the Receive function on the +// returned instance. +// +// See LoadWallet for the blocking version and more details. +func (c *Client) LoadWalletAsync(walletName string) FutureLoadWalletResult { + return c.sendCmd(btcjson.NewLoadWalletCmd(walletName)) +} + +// LoadWallet loads a wallet from a wallet file or directory. +func (c *Client) LoadWallet(walletName string) (*btcjson.LoadWalletResult, error) { + return c.LoadWalletAsync(walletName).Receive() +} + // TODO(davec): Implement -// backupwallet (NYI in btcwallet) // encryptwallet (Won't be supported by btcwallet since it's always encrypted) -// getwalletinfo (NYI in btcwallet or btcjson) // listaddressgroupings (NYI in btcwallet) // listreceivedbyaccount (NYI in btcwallet) - -// DUMP -// importwallet (NYI in btcwallet) -// dumpwallet (NYI in btcwallet) diff --git a/rpcserver.go b/rpcserver.go index 89817db856..d184072942 100644 --- a/rpcserver.go +++ b/rpcserver.go @@ -101,6 +101,9 @@ var ( // declared here to avoid the overhead of creating the slice on every // invocation for constant data. gbtCapabilities = []string{"proposal"} + + // JSON 2.0 batched request prefix + batchedRequestPrefix = []byte("[") ) // Errors @@ -127,52 +130,54 @@ type commandHandler func(*rpcServer, interface{}, <-chan struct{}) (interface{}, // a dependency loop. var rpcHandlers map[string]commandHandler var rpcHandlersBeforeInit = map[string]commandHandler{ - "addnode": handleAddNode, - "createrawtransaction": handleCreateRawTransaction, - "debuglevel": handleDebugLevel, - "decoderawtransaction": handleDecodeRawTransaction, - "decodescript": handleDecodeScript, - "estimatefee": handleEstimateFee, - "generate": handleGenerate, - "getaddednodeinfo": handleGetAddedNodeInfo, - "getbestblock": handleGetBestBlock, - "getbestblockhash": handleGetBestBlockHash, - "getblock": handleGetBlock, - "getblockchaininfo": handleGetBlockChainInfo, - "getblockcount": handleGetBlockCount, - "getblockhash": handleGetBlockHash, - "getblockheader": handleGetBlockHeader, - "getblocktemplate": handleGetBlockTemplate, - "getcfilter": handleGetCFilter, - "getcfilterheader": handleGetCFilterHeader, - "getconnectioncount": handleGetConnectionCount, - "getcurrentnet": handleGetCurrentNet, - "getdifficulty": handleGetDifficulty, - "getgenerate": handleGetGenerate, - "gethashespersec": handleGetHashesPerSec, - "getheaders": handleGetHeaders, - "getinfo": handleGetInfo, - "getmempoolinfo": handleGetMempoolInfo, - "getmininginfo": handleGetMiningInfo, - "getnettotals": handleGetNetTotals, - "getnetworkhashps": handleGetNetworkHashPS, - "getpeerinfo": handleGetPeerInfo, - "getrawmempool": handleGetRawMempool, - "getrawtransaction": handleGetRawTransaction, - "gettxout": handleGetTxOut, - "help": handleHelp, - "node": handleNode, - "ping": handlePing, - "searchrawtransactions": handleSearchRawTransactions, - "sendrawtransaction": handleSendRawTransaction, - "setgenerate": handleSetGenerate, - "stop": handleStop, - "submitblock": handleSubmitBlock, - "uptime": handleUptime, - "validateaddress": handleValidateAddress, - "verifychain": handleVerifyChain, - "verifymessage": handleVerifyMessage, - "version": handleVersion, + "addnode": handleAddNode, + "createrawtransaction": handleCreateRawTransaction, + "debuglevel": handleDebugLevel, + "decoderawtransaction": handleDecodeRawTransaction, + "decodescript": handleDecodeScript, + "estimatefee": handleEstimateFee, + "generate": handleGenerate, + "getaddednodeinfo": handleGetAddedNodeInfo, + "getbestblock": handleGetBestBlock, + "getbestblockhash": handleGetBestBlockHash, + "getblock": handleGetBlock, + "getblockchaininfo": handleGetBlockChainInfo, + "getblockcount": handleGetBlockCount, + "getblockhash": handleGetBlockHash, + "getblockheader": handleGetBlockHeader, + "getblocktemplate": handleGetBlockTemplate, + "getcfilter": handleGetCFilter, + "getcfilterheader": handleGetCFilterHeader, + "getconnectioncount": handleGetConnectionCount, + "getcurrentnet": handleGetCurrentNet, + "getdifficulty": handleGetDifficulty, + "getgenerate": handleGetGenerate, + "gethashespersec": handleGetHashesPerSec, + "getheaders": handleGetHeaders, + "getinfo": handleGetInfo, + "getmempoolinfo": handleGetMempoolInfo, + "getmininginfo": handleGetMiningInfo, + "getnettotals": handleGetNetTotals, + "getnetworkhashps": handleGetNetworkHashPS, + "getnodeaddresses": handleGetNodeAddresses, + "getpeerinfo": handleGetPeerInfo, + "getrawmempool": handleGetRawMempool, + "getrawtransaction": handleGetRawTransaction, + "gettxout": handleGetTxOut, + "help": handleHelp, + "node": handleNode, + "ping": handlePing, + "searchrawtransactions": handleSearchRawTransactions, + "sendrawtransaction": handleSendRawTransaction, + "setgenerate": handleSetGenerate, + "signmessagewithprivkey": handleSignMessageWithPrivKey, + "stop": handleStop, + "submitblock": handleSubmitBlock, + "uptime": handleUptime, + "validateaddress": handleValidateAddress, + "verifychain": handleVerifyChain, + "verifymessage": handleVerifyMessage, + "version": handleVersion, } // list of commands that we recognize, but for which btcd has no support because @@ -543,7 +548,7 @@ func handleCreateRawTransaction(s *rpcServer, cmd interface{}, closeChan <-chan params := s.cfg.ChainParams for encodedAddr, amount := range c.Amounts { // Ensure amount is in the valid range for monetary amounts. - if amount <= 0 || amount > btcutil.MaxSatoshi { + if amount <= 0 || amount*btcutil.SatoshiPerBitcoin > btcutil.MaxSatoshi { return nil, &btcjson.RPCError{ Code: btcjson.ErrRPCType, Message: "Invalid amount", @@ -759,7 +764,7 @@ func createTxRawResult(chainParams *chaincfg.Params, mtx *wire.MsgTx, Weight: int32(blockchain.GetTransactionWeight(btcutil.NewTx(mtx))), Vin: createVinList(mtx), Vout: createVoutList(mtx, chainParams, nil), - Version: mtx.Version, + Version: uint32(mtx.Version), LockTime: mtx.LockTime, } @@ -1251,6 +1256,9 @@ func handleGetBlockChainInfo(s *rpcServer, cmd interface{}, closeChan <-chan str case chaincfg.DeploymentSegwit: forkName = "segwit" + case chaincfg.DeploymentTaproot: + forkName = "taproot" + default: return nil, &btcjson.RPCError{ Code: btcjson.ErrRPCInternal.Code, @@ -1687,8 +1695,8 @@ func (state *gbtWorkState) blockTemplateResult(useCoinbaseValue bool, submitOld transactions := make([]btcjson.GetBlockTemplateResultTx, 0, numTx-1) txIndex := make(map[chainhash.Hash]int64, numTx) for i, tx := range msgBlock.Transactions { - txHash := tx.TxHash() - txIndex[txHash] = int64(i) + txID := tx.TxHash() + txIndex[txID] = int64(i) // Skip the coinbase transaction. if i == 0 { @@ -1722,7 +1730,8 @@ func (state *gbtWorkState) blockTemplateResult(useCoinbaseValue bool, submitOld bTx := btcutil.NewTx(tx) resultTx := btcjson.GetBlockTemplateResultTx{ Data: hex.EncodeToString(txBuf.Bytes()), - Hash: txHash.String(), + TxID: txID.String(), + Hash: tx.WitnessHash().String(), Depends: depends, Fee: template.Fees[i], SigOps: template.SigOpCosts[i], @@ -2363,8 +2372,8 @@ func handleGetMiningInfo(s *rpcServer, cmd interface{}, closeChan <-chan struct{ Difficulty: getDifficultyRatio(best.Bits, s.cfg.ChainParams), Generate: s.cfg.CPUMiner.IsMining(), GenProcLimit: s.cfg.CPUMiner.NumWorkers(), - HashesPerSec: int64(s.cfg.CPUMiner.HashesPerSecond()), - NetworkHashPS: networkHashesPerSec, + HashesPerSec: s.cfg.CPUMiner.HashesPerSecond(), + NetworkHashPS: float64(networkHashesPerSec), PooledTx: uint64(s.cfg.TxMemPool.Count()), TestNet: cfg.TestNet3, } @@ -2476,6 +2485,40 @@ func handleGetNetworkHashPS(s *rpcServer, cmd interface{}, closeChan <-chan stru return hashesPerSec.Int64(), nil } +// handleGetNodeAddresses implements the getnodeaddresses command. +func handleGetNodeAddresses(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) (interface{}, error) { + c := cmd.(*btcjson.GetNodeAddressesCmd) + + count := int32(1) + if c.Count != nil { + count = *c.Count + if count <= 0 { + return nil, &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidParameter, + Message: "Address count out of range", + } + } + } + + nodes := s.cfg.ConnMgr.NodeAddresses() + if n := int32(len(nodes)); n < count { + count = n + } + + addresses := make([]*btcjson.GetNodeAddressesResult, 0, count) + for _, node := range nodes[:count] { + address := &btcjson.GetNodeAddressesResult{ + Time: node.Timestamp.Unix(), + Services: uint64(node.Services), + Address: node.IP.String(), + Port: node.Port, + } + addresses = append(addresses, address) + } + + return addresses, nil +} + // handleGetPeerInfo implements the getpeerinfo command. func handleGetPeerInfo(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) (interface{}, error) { peers := s.cfg.ConnMgr.ConnectedPeers() @@ -3435,6 +3478,52 @@ func handleSetGenerate(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) return nil, nil } +// Text used to signify that a signed message follows and to prevent +// inadvertently signing a transaction. +const messageSignatureHeader = "Bitcoin Signed Message:\n" + +// handleSignMessageWithPrivKey implements the signmessagewithprivkey command. +func handleSignMessageWithPrivKey(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) (interface{}, error) { + c := cmd.(*btcjson.SignMessageWithPrivKeyCmd) + + wif, err := btcutil.DecodeWIF(c.PrivKey) + if err != nil { + message := "Invalid private key" + switch err { + case btcutil.ErrMalformedPrivateKey: + message = "Malformed private key" + case btcutil.ErrChecksumMismatch: + message = "Private key checksum mismatch" + } + return nil, &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidAddressOrKey, + Message: message, + } + } + if !wif.IsForNet(s.cfg.ChainParams) { + return nil, &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidAddressOrKey, + Message: "Private key for wrong network", + } + } + + var buf bytes.Buffer + wire.WriteVarString(&buf, 0, messageSignatureHeader) + wire.WriteVarString(&buf, 0, c.Message) + messageHash := chainhash.DoubleHashB(buf.Bytes()) + + sig, err := btcec.SignCompact(btcec.S256(), wif.PrivKey, + messageHash, wif.CompressPubKey) + if err != nil { + return nil, &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidAddressOrKey, + Message: "Sign failed", + } + } + + return base64.StdEncoding.EncodeToString(sig), nil +} + // handleStop implements the stop command. func handleStop(s *rpcServer, cmd interface{}, closeChan <-chan struct{}) (interface{}, error) { select { @@ -3493,6 +3582,37 @@ func handleValidateAddress(s *rpcServer, cmd interface{}, closeChan <-chan struc return result, nil } + switch addr := addr.(type) { + case *btcutil.AddressPubKeyHash: + result.IsScript = btcjson.Bool(false) + result.IsWitness = btcjson.Bool(false) + + case *btcutil.AddressScriptHash: + result.IsScript = btcjson.Bool(true) + result.IsWitness = btcjson.Bool(false) + + case *btcutil.AddressPubKey: + result.IsScript = btcjson.Bool(false) + result.IsWitness = btcjson.Bool(false) + + case *btcutil.AddressWitnessPubKeyHash: + result.IsScript = btcjson.Bool(false) + result.IsWitness = btcjson.Bool(true) + result.WitnessVersion = btcjson.Int32(int32(addr.WitnessVersion())) + result.WitnessProgram = btcjson.String(hex.EncodeToString(addr.WitnessProgram())) + + case *btcutil.AddressWitnessScriptHash: + result.IsScript = btcjson.Bool(true) + result.IsWitness = btcjson.Bool(true) + result.WitnessVersion = btcjson.Int32(int32(addr.WitnessVersion())) + result.WitnessProgram = btcjson.String(hex.EncodeToString(addr.WitnessProgram())) + + default: + // Handle the case when a new Address is supported by btcutil, but none + // of the cases were matched in the switch block. The current behaviour + // is to do nothing, and only populate the Address and IsValid fields. + } + result.Address = addr.EncodeAddress() result.IsValid = true @@ -3584,7 +3704,7 @@ func handleVerifyMessage(s *rpcServer, cmd interface{}, closeChan <-chan struct{ // Validate the signature - this just shows that it was valid at all. // we will compare it with the key next. var buf bytes.Buffer - wire.WriteVarString(&buf, 0, "Bitcoin Signed Message:\n") + wire.WriteVarString(&buf, 0, messageSignatureHeader) wire.WriteVarString(&buf, 0, c.Message) expectedMessageHash := chainhash.DoubleHashB(buf.Bytes()) pk, wasCompressed, err := btcec.RecoverCompact(btcec.S256(), sig, @@ -3825,10 +3945,11 @@ func (s *rpcServer) checkAuth(r *http.Request, require bool) (bool, bool, error) // a known concrete command along with any error that might have happened while // parsing it. type parsedRPCCmd struct { - id interface{} - method string - cmd interface{} - err *btcjson.RPCError + jsonrpc btcjson.RPCVersion + id interface{} + method string + cmd interface{} + err *btcjson.RPCError } // standardCmdResult checks that a parsed command is a standard Bitcoin JSON-RPC @@ -3861,9 +3982,11 @@ handled: // is suitable for use in replies if the command is invalid in some way such as // an unregistered command or invalid parameters. func parseCmd(request *btcjson.Request) *parsedRPCCmd { - var parsedCmd parsedRPCCmd - parsedCmd.id = request.ID - parsedCmd.method = request.Method + parsedCmd := parsedRPCCmd{ + jsonrpc: request.Jsonrpc, + id: request.ID, + method: request.Method, + } cmd, err := btcjson.UnmarshalCmd(request) if err != nil { @@ -3890,7 +4013,7 @@ func parseCmd(request *btcjson.Request) *parsedRPCCmd { // createMarshalledReply returns a new marshalled JSON-RPC response given the // passed parameters. It will automatically convert errors that are not of // the type *btcjson.RPCError to the appropriate type as needed. -func createMarshalledReply(id, result interface{}, replyErr error) ([]byte, error) { +func createMarshalledReply(rpcVersion btcjson.RPCVersion, id interface{}, result interface{}, replyErr error) ([]byte, error) { var jsonErr *btcjson.RPCError if replyErr != nil { if jErr, ok := replyErr.(*btcjson.RPCError); ok { @@ -3900,7 +4023,71 @@ func createMarshalledReply(id, result interface{}, replyErr error) ([]byte, erro } } - return btcjson.MarshalResponse(id, result, jsonErr) + return btcjson.MarshalResponse(rpcVersion, id, result, jsonErr) +} + +// processRequest determines the incoming request type (single or batched), +// parses it and returns a marshalled response. +func (s *rpcServer) processRequest(request *btcjson.Request, isAdmin bool, closeChan <-chan struct{}) []byte { + var result interface{} + var err error + var jsonErr *btcjson.RPCError + + if !isAdmin { + if _, ok := rpcLimited[request.Method]; !ok { + jsonErr = internalRPCError("limited user not "+ + "authorized for this method", "") + } + } + + if jsonErr == nil { + if request.Method == "" || request.Params == nil { + jsonErr = &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: malformed", + } + msg, err := createMarshalledReply(request.Jsonrpc, request.ID, result, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + return nil + } + return msg + } + + // Valid requests with no ID (notifications) must not have a response + // per the JSON-RPC spec. + if request.ID == nil { + return nil + } + + // Attempt to parse the JSON-RPC request into a known + // concrete command. + parsedCmd := parseCmd(request) + if parsedCmd.err != nil { + jsonErr = parsedCmd.err + } else { + result, err = s.standardCmdResult(parsedCmd, + closeChan) + if err != nil { + if rpcErr, ok := err.(*btcjson.RPCError); ok { + jsonErr = rpcErr + } else { + jsonErr = &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: malformed", + } + } + } + } + } + + // Marshal the response. + msg, err := createMarshalledReply(request.Jsonrpc, request.ID, result, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + return nil + } + return msg } // jsonRPCRead handles reading and responding to RPC messages. @@ -3945,80 +4132,186 @@ func (s *rpcServer) jsonRPCRead(w http.ResponseWriter, r *http.Request, isAdmin conn.SetReadDeadline(timeZeroVal) // Attempt to parse the raw body into a JSON-RPC request. - var responseID interface{} - var jsonErr error - var result interface{} - var request btcjson.Request - if err := json.Unmarshal(body, &request); err != nil { - jsonErr = &btcjson.RPCError{ - Code: btcjson.ErrRPCParse.Code, - Message: "Failed to parse request: " + err.Error(), + // Setup a close notifier. Since the connection is hijacked, + // the CloseNotifer on the ResponseWriter is not available. + closeChan := make(chan struct{}, 1) + go func() { + _, err = conn.Read(make([]byte, 1)) + if err != nil { + close(closeChan) } + }() + + var results []json.RawMessage + var batchSize int + var batchedRequest bool + + // Determine request type + if bytes.HasPrefix(body, batchedRequestPrefix) { + batchedRequest = true } - if jsonErr == nil { - // The JSON-RPC 1.0 spec defines that notifications must have their "id" - // set to null and states that notifications do not have a response. - // - // A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and - // without an "id" member. The specification states that notifications - // must not be responded to. JSON-RPC 2.0 permits the null value as a - // valid request id, therefore such requests are not notifications. - // - // Bitcoin Core serves requests with "id":null or even an absent "id", - // and responds to such requests with "id":null in the response. - // - // Btcd does not respond to any request without and "id" or "id":null, - // regardless the indicated JSON-RPC protocol version unless RPC quirks - // are enabled. With RPC quirks enabled, such requests will be responded - // to if the reqeust does not indicate JSON-RPC version. - // - // RPC quirks can be enabled by the user to avoid compatibility issues - // with software relying on Core's behavior. - if request.ID == nil && !(cfg.RPCQuirks && request.Jsonrpc == "") { - return + + // Process a single request + if !batchedRequest { + var req btcjson.Request + var resp json.RawMessage + err = json.Unmarshal(body, &req) + if err != nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCParse.Code, + Message: fmt.Sprintf("Failed to parse request: %v", + err), + } + resp, err = btcjson.MarshalResponse(btcjson.RpcVersion1, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + } + } + + if err == nil { + // The JSON-RPC 1.0 spec defines that notifications must have their "id" + // set to null and states that notifications do not have a response. + // + // A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and + // without an "id" member. The specification states that notifications + // must not be responded to. JSON-RPC 2.0 permits the null value as a + // valid request id, therefore such requests are not notifications. + // + // Bitcoin Core serves requests with "id":null or even an absent "id", + // and responds to such requests with "id":null in the response. + // + // Btcd does not respond to any request without and "id" or "id":null, + // regardless the indicated JSON-RPC protocol version unless RPC quirks + // are enabled. With RPC quirks enabled, such requests will be responded + // to if the reqeust does not indicate JSON-RPC version. + // + // RPC quirks can be enabled by the user to avoid compatibility issues + // with software relying on Core's behavior. + if req.ID == nil && !(cfg.RPCQuirks && req.Jsonrpc == "") { + return + } + resp = s.processRequest(&req, isAdmin, closeChan) } - // The parse was at least successful enough to have an ID so - // set it for the response. - responseID = request.ID + if resp != nil { + results = append(results, resp) + } + } - // Setup a close notifier. Since the connection is hijacked, - // the CloseNotifer on the ResponseWriter is not available. - closeChan := make(chan struct{}, 1) - go func() { - _, err := conn.Read(make([]byte, 1)) + // Process a batched request + if batchedRequest { + var batchedRequests []interface{} + var resp json.RawMessage + err = json.Unmarshal(body, &batchedRequests) + if err != nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCParse.Code, + Message: fmt.Sprintf("Failed to parse request: %v", + err), + } + resp, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) if err != nil { - close(closeChan) + rpcsLog.Errorf("Failed to create reply: %v", err) + } + + if resp != nil { + results = append(results, resp) + } + } + + if err == nil { + // Response with an empty batch error if the batch size is zero + if len(batchedRequests) == 0 { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: empty batch", + } + resp, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + } + + if resp != nil { + results = append(results, resp) + } } - }() - - // Check if the user is limited and set error if method unauthorized - if !isAdmin { - if _, ok := rpcLimited[request.Method]; !ok { - jsonErr = &btcjson.RPCError{ - Code: btcjson.ErrRPCInvalidParams.Code, - Message: "limited user not authorized for this method", + + // Process each batch entry individually + if len(batchedRequests) > 0 { + batchSize = len(batchedRequests) + + for _, entry := range batchedRequests { + var reqBytes []byte + reqBytes, err = json.Marshal(entry) + if err != nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: fmt.Sprintf("Invalid request: %v", + err), + } + resp, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + } + + if resp != nil { + results = append(results, resp) + } + continue + } + + var req btcjson.Request + err := json.Unmarshal(reqBytes, &req) + if err != nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: fmt.Sprintf("Invalid request: %v", + err), + } + resp, err = btcjson.MarshalResponse("", nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + } + + if resp != nil { + results = append(results, resp) + } + continue + } + + resp = s.processRequest(&req, isAdmin, closeChan) + if resp != nil { + results = append(results, resp) + } } } } + } - if jsonErr == nil { - // Attempt to parse the JSON-RPC request into a known concrete - // command. - parsedCmd := parseCmd(&request) - if parsedCmd.err != nil { - jsonErr = parsedCmd.err - } else { - result, jsonErr = s.standardCmdResult(parsedCmd, closeChan) + var msg = []byte{} + if batchedRequest && batchSize > 0 { + if len(results) > 0 { + // Form the batched response json + var buffer bytes.Buffer + buffer.WriteByte('[') + for idx, reply := range results { + if idx == len(results)-1 { + buffer.Write(reply) + buffer.WriteByte(']') + break + } + buffer.Write(reply) + buffer.WriteByte(',') } + msg = buffer.Bytes() } } - // Marshal the response. - msg, err := createMarshalledReply(responseID, result, jsonErr) - if err != nil { - rpcsLog.Errorf("Failed to marshal reply: %v", err) - return + if !batchedRequest || batchSize == 0 { + // Respond with the first results entry for single requests + if len(results) > 0 { + msg = results[0] + } } // Write the response. @@ -4220,6 +4513,10 @@ type rpcserverConnManager interface { // RelayTransactions generates and relays inventory vectors for all of // the passed transactions to all connected peers. RelayTransactions(txns []*mempool.TxDesc) + + // NodeAddresses returns an array consisting node addresses which can + // potentially be used to find new nodes in the network. + NodeAddresses() []*wire.NetAddress } // rpcserverSyncManager represents a sync manager for use with the RPC server. diff --git a/rpcserverhelp.go b/rpcserverhelp.go index cc3b33f3ce..654fee0169 100644 --- a/rpcserverhelp.go +++ b/rpcserverhelp.go @@ -181,6 +181,8 @@ var helpDescsEnUS = map[string]string{ "getblockchaininforesult-pruned": "A bool that indicates if the node is pruned or not", "getblockchaininforesult-pruneheight": "The lowest block retained in the current pruned chain", "getblockchaininforesult-chainwork": "The total cumulative work in the best chain", + "getblockchaininforesult-size_on_disk": "The estimated size of the block and undo files on disk", + "getblockchaininforesult-initialblockdownload": "Estimate of whether this node is in Initial Block Download mode", "getblockchaininforesult-softforks": "The status of the super-majority soft-forks", "getblockchaininforesult-unifiedsoftforks": "The status of the super-majority soft-forks used by bitcoind on or after v0.19.0", @@ -295,6 +297,7 @@ var helpDescsEnUS = map[string]string{ "templaterequest-target": "The desired target for the block template (this parameter is ignored)", "templaterequest-data": "Hex-encoded block data (only for mode=proposal)", "templaterequest-workid": "The server provided workid if provided in block template (not applicable)", + "templaterequest-rules": "Specific block rules that are to be enforced e.g. '[\"segwit\"]", // GetBlockTemplateResultTx help. "getblocktemplateresulttx-data": "Hex-encoded transaction data (byte-for-byte)", @@ -302,6 +305,7 @@ var helpDescsEnUS = map[string]string{ "getblocktemplateresulttx-depends": "Other transactions before this one (by 1-based index in the 'transactions' list) that must be present in the final block if this one is", "getblocktemplateresulttx-fee": "Difference in value between transaction inputs and outputs (in Satoshi)", "getblocktemplateresulttx-sigops": "Total number of signature operations as counted for purposes of block limits", + "getblocktemplateresulttx-txid": "The transaction id, can be different from hash.", "getblocktemplateresulttx-weight": "The weight of the transaction", // GetBlockTemplateResultAux help. @@ -452,6 +456,17 @@ var helpDescsEnUS = map[string]string{ "getnettotalsresult-totalbytessent": "Total bytes sent", "getnettotalsresult-timemillis": "Number of milliseconds since 1 Jan 1970 GMT", + // GetNodeAddressesResult help. + "getnodeaddressesresult-time": "Timestamp in seconds since epoch (Jan 1 1970 GMT) keeping track of when the node was last seen", + "getnodeaddressesresult-services": "The services offered", + "getnodeaddressesresult-address": "The address of the node", + "getnodeaddressesresult-port": "The port of the node", + + // GetNodeAddressesCmd help. + "getnodeaddresses--synopsis": "Return known addresses which can potentially be used to find new nodes in the network", + "getnodeaddresses-count": "How many addresses to return. Limited to the smaller of 2500 or 23% of all known addresses", + "getnodeaddresses--result0": "List of node addresses", + // GetPeerInfoResult help. "getpeerinforesult-id": "A unique node ID", "getpeerinforesult-addr": "The ip address and port of the peer", @@ -548,17 +563,23 @@ var helpDescsEnUS = map[string]string{ "searchrawtransactions--result0": "Hex-encoded serialized transaction", // SendRawTransactionCmd help. - "sendrawtransaction--synopsis": "Submits the serialized, hex-encoded transaction to the local peer and relays it to the network.", - "sendrawtransaction-hextx": "Serialized, hex-encoded signed transaction", - "sendrawtransaction-allowhighfees": "Whether or not to allow insanely high fees (btcd does not yet implement this parameter, so it has no effect)", - "sendrawtransaction-maxfeerate": "Used by bitcoind on or after v0.19.0", - "sendrawtransaction--result0": "The hash of the transaction", + "sendrawtransaction--synopsis": "Submits the serialized, hex-encoded transaction to the local peer and relays it to the network.", + "sendrawtransaction-hextx": "Serialized, hex-encoded signed transaction", + "sendrawtransaction-feesetting": "Whether or not to allow insanely high fees in bitcoind < v0.19.0 or the max fee rate for bitcoind v0.19.0 and later (btcd does not yet implement this parameter, so it has no effect)", + "sendrawtransaction--result0": "The hash of the transaction", + "allowhighfeesormaxfeerate-value": "Either the boolean value for the allowhighfees parameter in bitcoind < v0.19.0 or the numerical value for the maxfeerate field in bitcoind v0.19.0 and later", // SetGenerateCmd help. "setgenerate--synopsis": "Set the server to generate coins (mine) or not.", "setgenerate-generate": "Use true to enable generation, false to disable it", "setgenerate-genproclimit": "The number of processors (cores) to limit generation to or -1 for default", + // SignMessageWithPrivKeyCmd help. + "signmessagewithprivkey--synopsis": "Sign a message with the private key of an address", + "signmessagewithprivkey-privkey": "The private key to sign the message with", + "signmessagewithprivkey-message": "The message to create a signature of", + "signmessagewithprivkey--result0": "The signature of the message encoded in base 64", + // StopCmd help. "stop--synopsis": "Shutdown btcd.", "stop--result0": "The string 'btcd stopping.'", @@ -575,8 +596,12 @@ var helpDescsEnUS = map[string]string{ "submitblock--result1": "The reason the block was rejected", // ValidateAddressResult help. - "validateaddresschainresult-isvalid": "Whether or not the address is valid", - "validateaddresschainresult-address": "The bitcoin address (only when isvalid is true)", + "validateaddresschainresult-isvalid": "Whether or not the address is valid", + "validateaddresschainresult-address": "The bitcoin address (only when isvalid is true)", + "validateaddresschainresult-isscript": "If the key is a script", + "validateaddresschainresult-iswitness": "If the address is a witness address", + "validateaddresschainresult-witness_version": "The version number of the witness program", + "validateaddresschainresult-witness_program": "The hex value of the witness program", // ValidateAddressCmd help. "validateaddress--synopsis": "Verify an address is valid.", @@ -687,52 +712,54 @@ var helpDescsEnUS = map[string]string{ // This information is used to generate the help. Each result type must be a // pointer to the type (or nil to indicate no return value). var rpcResultTypes = map[string][]interface{}{ - "addnode": nil, - "createrawtransaction": {(*string)(nil)}, - "debuglevel": {(*string)(nil), (*string)(nil)}, - "decoderawtransaction": {(*btcjson.TxRawDecodeResult)(nil)}, - "decodescript": {(*btcjson.DecodeScriptResult)(nil)}, - "estimatefee": {(*float64)(nil)}, - "generate": {(*[]string)(nil)}, - "getaddednodeinfo": {(*[]string)(nil), (*[]btcjson.GetAddedNodeInfoResult)(nil)}, - "getbestblock": {(*btcjson.GetBestBlockResult)(nil)}, - "getbestblockhash": {(*string)(nil)}, - "getblock": {(*string)(nil), (*btcjson.GetBlockVerboseResult)(nil)}, - "getblockcount": {(*int64)(nil)}, - "getblockhash": {(*string)(nil)}, - "getblockheader": {(*string)(nil), (*btcjson.GetBlockHeaderVerboseResult)(nil)}, - "getblocktemplate": {(*btcjson.GetBlockTemplateResult)(nil), (*string)(nil), nil}, - "getblockchaininfo": {(*btcjson.GetBlockChainInfoResult)(nil)}, - "getcfilter": {(*string)(nil)}, - "getcfilterheader": {(*string)(nil)}, - "getconnectioncount": {(*int32)(nil)}, - "getcurrentnet": {(*uint32)(nil)}, - "getdifficulty": {(*float64)(nil)}, - "getgenerate": {(*bool)(nil)}, - "gethashespersec": {(*float64)(nil)}, - "getheaders": {(*[]string)(nil)}, - "getinfo": {(*btcjson.InfoChainResult)(nil)}, - "getmempoolinfo": {(*btcjson.GetMempoolInfoResult)(nil)}, - "getmininginfo": {(*btcjson.GetMiningInfoResult)(nil)}, - "getnettotals": {(*btcjson.GetNetTotalsResult)(nil)}, - "getnetworkhashps": {(*int64)(nil)}, - "getpeerinfo": {(*[]btcjson.GetPeerInfoResult)(nil)}, - "getrawmempool": {(*[]string)(nil), (*btcjson.GetRawMempoolVerboseResult)(nil)}, - "getrawtransaction": {(*string)(nil), (*btcjson.TxRawResult)(nil)}, - "gettxout": {(*btcjson.GetTxOutResult)(nil)}, - "node": nil, - "help": {(*string)(nil), (*string)(nil)}, - "ping": nil, - "searchrawtransactions": {(*string)(nil), (*[]btcjson.SearchRawTransactionsResult)(nil)}, - "sendrawtransaction": {(*string)(nil)}, - "setgenerate": nil, - "stop": {(*string)(nil)}, - "submitblock": {nil, (*string)(nil)}, - "uptime": {(*int64)(nil)}, - "validateaddress": {(*btcjson.ValidateAddressChainResult)(nil)}, - "verifychain": {(*bool)(nil)}, - "verifymessage": {(*bool)(nil)}, - "version": {(*map[string]btcjson.VersionResult)(nil)}, + "addnode": nil, + "createrawtransaction": {(*string)(nil)}, + "debuglevel": {(*string)(nil), (*string)(nil)}, + "decoderawtransaction": {(*btcjson.TxRawDecodeResult)(nil)}, + "decodescript": {(*btcjson.DecodeScriptResult)(nil)}, + "estimatefee": {(*float64)(nil)}, + "generate": {(*[]string)(nil)}, + "getaddednodeinfo": {(*[]string)(nil), (*[]btcjson.GetAddedNodeInfoResult)(nil)}, + "getbestblock": {(*btcjson.GetBestBlockResult)(nil)}, + "getbestblockhash": {(*string)(nil)}, + "getblock": {(*string)(nil), (*btcjson.GetBlockVerboseResult)(nil)}, + "getblockcount": {(*int64)(nil)}, + "getblockhash": {(*string)(nil)}, + "getblockheader": {(*string)(nil), (*btcjson.GetBlockHeaderVerboseResult)(nil)}, + "getblocktemplate": {(*btcjson.GetBlockTemplateResult)(nil), (*string)(nil), nil}, + "getblockchaininfo": {(*btcjson.GetBlockChainInfoResult)(nil)}, + "getcfilter": {(*string)(nil)}, + "getcfilterheader": {(*string)(nil)}, + "getconnectioncount": {(*int32)(nil)}, + "getcurrentnet": {(*uint32)(nil)}, + "getdifficulty": {(*float64)(nil)}, + "getgenerate": {(*bool)(nil)}, + "gethashespersec": {(*float64)(nil)}, + "getheaders": {(*[]string)(nil)}, + "getinfo": {(*btcjson.InfoChainResult)(nil)}, + "getmempoolinfo": {(*btcjson.GetMempoolInfoResult)(nil)}, + "getmininginfo": {(*btcjson.GetMiningInfoResult)(nil)}, + "getnettotals": {(*btcjson.GetNetTotalsResult)(nil)}, + "getnetworkhashps": {(*int64)(nil)}, + "getnodeaddresses": {(*[]btcjson.GetNodeAddressesResult)(nil)}, + "getpeerinfo": {(*[]btcjson.GetPeerInfoResult)(nil)}, + "getrawmempool": {(*[]string)(nil), (*btcjson.GetRawMempoolVerboseResult)(nil)}, + "getrawtransaction": {(*string)(nil), (*btcjson.TxRawResult)(nil)}, + "gettxout": {(*btcjson.GetTxOutResult)(nil)}, + "node": nil, + "help": {(*string)(nil), (*string)(nil)}, + "ping": nil, + "searchrawtransactions": {(*string)(nil), (*[]btcjson.SearchRawTransactionsResult)(nil)}, + "sendrawtransaction": {(*string)(nil)}, + "setgenerate": nil, + "signmessagewithprivkey": {(*string)(nil)}, + "stop": {(*string)(nil)}, + "submitblock": {nil, (*string)(nil)}, + "uptime": {(*int64)(nil)}, + "validateaddress": {(*btcjson.ValidateAddressChainResult)(nil)}, + "verifychain": {(*bool)(nil)}, + "verifymessage": {(*bool)(nil)}, + "version": {(*map[string]btcjson.VersionResult)(nil)}, // Websocket commands. "loadtxfilter": nil, diff --git a/rpcwebsocket.go b/rpcwebsocket.go index 32e466d115..356a897455 100644 --- a/rpcwebsocket.go +++ b/rpcwebsocket.go @@ -695,7 +695,7 @@ func (*wsNotificationManager) notifyBlockConnected(clients map[chan struct{}]*ws // Notify interested websocket clients about the connected block. ntfn := btcjson.NewBlockConnectedNtfn(block.Hash().String(), block.Height(), block.MsgBlock().Header.Timestamp.Unix()) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal block connected notification: "+ "%v", err) @@ -719,7 +719,7 @@ func (*wsNotificationManager) notifyBlockDisconnected(clients map[chan struct{}] // Notify interested websocket clients about the disconnected block. ntfn := btcjson.NewBlockDisconnectedNtfn(block.Hash().String(), block.Height(), block.MsgBlock().Header.Timestamp.Unix()) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal block disconnected "+ "notification: %v", err) @@ -765,7 +765,7 @@ func (m *wsNotificationManager) notifyFilteredBlockConnected(clients map[chan st ntfn.SubscribedTxs = subscribedTxs[quitChan] // Marshal and queue notification. - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal filtered block "+ "connected notification: %v", err) @@ -796,7 +796,7 @@ func (*wsNotificationManager) notifyFilteredBlockDisconnected(clients map[chan s } ntfn := btcjson.NewFilteredBlockDisconnectedNtfn(block.Height(), hex.EncodeToString(w.Bytes())) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal filtered block disconnected "+ "notification: %v", err) @@ -831,7 +831,7 @@ func (m *wsNotificationManager) notifyForNewTx(clients map[chan struct{}]*wsClie } ntfn := btcjson.NewTxAcceptedNtfn(txHashStr, btcutil.Amount(amount).ToBTC()) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal tx notification: %s", err.Error()) return @@ -854,7 +854,7 @@ func (m *wsNotificationManager) notifyForNewTx(clients map[chan struct{}]*wsClie } verboseNtfn = btcjson.NewTxAcceptedVerboseNtfn(*rawTx) - marshalledJSONVerbose, err = btcjson.MarshalCmd(nil, + marshalledJSONVerbose, err = btcjson.MarshalCmd(btcjson.RpcVersion1, nil, verboseNtfn) if err != nil { rpcsLog.Errorf("Failed to marshal verbose tx "+ @@ -980,7 +980,7 @@ func blockDetails(block *btcutil.Block, txIndex int) *btcjson.BlockDetails { func newRedeemingTxNotification(txHex string, index int, block *btcutil.Block) ([]byte, error) { // Create and marshal the notification. ntfn := btcjson.NewRedeemingTxNtfn(txHex, blockDetails(block, index)) - return btcjson.MarshalCmd(nil, ntfn) + return btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) } // notifyForTxOuts examines each transaction output, notifying interested @@ -1016,7 +1016,7 @@ func (m *wsNotificationManager) notifyForTxOuts(ops map[wire.OutPoint]map[chan s ntfn := btcjson.NewRecvTxNtfn(txHex, blockDetails(block, tx.Index())) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal processedtx notification: %v", err) continue @@ -1047,7 +1047,7 @@ func (m *wsNotificationManager) notifyRelevantTxAccepted(tx *btcutil.Tx, if len(clientsToNotify) != 0 { n := btcjson.NewRelevantTxAcceptedNtfn(txHexString(tx.MsgTx())) - marshalled, err := btcjson.MarshalCmd(nil, n) + marshalled, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, n) if err != nil { rpcsLog.Errorf("Failed to marshal notification: %v", err) return @@ -1323,153 +1323,435 @@ out: break out } - var request btcjson.Request - err = json.Unmarshal(msg, &request) - if err != nil { - if !c.authenticated { - break out - } + var batchedRequest bool - jsonErr := &btcjson.RPCError{ - Code: btcjson.ErrRPCParse.Code, - Message: "Failed to parse request: " + err.Error(), - } - reply, err := createMarshalledReply(nil, nil, jsonErr) - if err != nil { - rpcsLog.Errorf("Failed to marshal parse failure "+ - "reply: %v", err) - continue - } - c.SendMessage(reply, nil) - continue + // Determine request type + if bytes.HasPrefix(msg, batchedRequestPrefix) { + batchedRequest = true } - // The JSON-RPC 1.0 spec defines that notifications must have their "id" - // set to null and states that notifications do not have a response. - // - // A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and - // without an "id" member. The specification states that notifications - // must not be responded to. JSON-RPC 2.0 permits the null value as a - // valid request id, therefore such requests are not notifications. - // - // Bitcoin Core serves requests with "id":null or even an absent "id", - // and responds to such requests with "id":null in the response. - // - // Btcd does not respond to any request without and "id" or "id":null, - // regardless the indicated JSON-RPC protocol version unless RPC quirks - // are enabled. With RPC quirks enabled, such requests will be responded - // to if the reqeust does not indicate JSON-RPC version. - // - // RPC quirks can be enabled by the user to avoid compatibility issues - // with software relying on Core's behavior. - if request.ID == nil && !(cfg.RPCQuirks && request.Jsonrpc == "") { - if !c.authenticated { - break out - } - continue - } + if !batchedRequest { + var req btcjson.Request + var reply json.RawMessage + err = json.Unmarshal(msg, &req) + if err != nil { + // only process requests from authenticated clients + if !c.authenticated { + break out + } - cmd := parseCmd(&request) - if cmd.err != nil { - if !c.authenticated { - break out + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCParse.Code, + Message: "Failed to parse request: " + err.Error(), + } + reply, err = createMarshalledReply(btcjson.RpcVersion1, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + continue + } + c.SendMessage(reply, nil) + continue } - reply, err := createMarshalledReply(cmd.id, nil, cmd.err) - if err != nil { - rpcsLog.Errorf("Failed to marshal parse failure "+ - "reply: %v", err) + if req.Method == "" || req.Params == nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: malformed", + } + reply, err := createMarshalledReply(req.Jsonrpc, req.ID, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + continue + } + c.SendMessage(reply, nil) continue } - c.SendMessage(reply, nil) - continue - } - rpcsLog.Debugf("Received command <%s> from %s", cmd.method, c.addr) - - // Check auth. The client is immediately disconnected if the - // first request of an unauthentiated websocket client is not - // the authenticate request, an authenticate request is received - // when the client is already authenticated, or incorrect - // authentication credentials are provided in the request. - switch authCmd, ok := cmd.cmd.(*btcjson.AuthenticateCmd); { - case c.authenticated && ok: - rpcsLog.Warnf("Websocket client %s is already authenticated", - c.addr) - break out - case !c.authenticated && !ok: - rpcsLog.Warnf("Unauthenticated websocket message " + - "received") - break out - case !c.authenticated: - // Check credentials. - login := authCmd.Username + ":" + authCmd.Passphrase - auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(login)) - authSha := sha256.Sum256([]byte(auth)) - cmp := subtle.ConstantTimeCompare(authSha[:], c.server.authsha[:]) - limitcmp := subtle.ConstantTimeCompare(authSha[:], c.server.limitauthsha[:]) - if cmp != 1 && limitcmp != 1 { - rpcsLog.Warnf("Auth failure.") - break out + + // Valid requests with no ID (notifications) must not have a response + // per the JSON-RPC spec. + if req.ID == nil { + if !c.authenticated { + break out + } + continue } - c.authenticated = true - c.isAdmin = cmp == 1 - // Marshal and send response. - reply, err := createMarshalledReply(cmd.id, nil, nil) - if err != nil { - rpcsLog.Errorf("Failed to marshal authenticate reply: "+ - "%v", err.Error()) + cmd := parseCmd(&req) + if cmd.err != nil { + // Only process requests from authenticated clients + if !c.authenticated { + break out + } + + reply, err = createMarshalledReply(cmd.jsonrpc, cmd.id, nil, cmd.err) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + continue + } + c.SendMessage(reply, nil) continue } - c.SendMessage(reply, nil) - continue - } - // Check if the client is using limited RPC credentials and - // error when not authorized to call this RPC. - if !c.isAdmin { - if _, ok := rpcLimited[request.Method]; !ok { - jsonErr := &btcjson.RPCError{ - Code: btcjson.ErrRPCInvalidParams.Code, - Message: "limited user not authorized for this method", + rpcsLog.Debugf("Received command <%s> from %s", cmd.method, c.addr) + + // Check auth. The client is immediately disconnected if the + // first request of an unauthentiated websocket client is not + // the authenticate request, an authenticate request is received + // when the client is already authenticated, or incorrect + // authentication credentials are provided in the request. + switch authCmd, ok := cmd.cmd.(*btcjson.AuthenticateCmd); { + case c.authenticated && ok: + rpcsLog.Warnf("Websocket client %s is already authenticated", + c.addr) + break out + case !c.authenticated && !ok: + rpcsLog.Warnf("Unauthenticated websocket message " + + "received") + break out + case !c.authenticated: + // Check credentials. + login := authCmd.Username + ":" + authCmd.Passphrase + auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(login)) + authSha := sha256.Sum256([]byte(auth)) + cmp := subtle.ConstantTimeCompare(authSha[:], c.server.authsha[:]) + limitcmp := subtle.ConstantTimeCompare(authSha[:], c.server.limitauthsha[:]) + if cmp != 1 && limitcmp != 1 { + rpcsLog.Warnf("Auth failure.") + break out } + c.authenticated = true + c.isAdmin = cmp == 1 + // Marshal and send response. - reply, err := createMarshalledReply(request.ID, nil, jsonErr) + reply, err = createMarshalledReply(cmd.jsonrpc, cmd.id, nil, nil) if err != nil { - rpcsLog.Errorf("Failed to marshal parse failure "+ - "reply: %v", err) + rpcsLog.Errorf("Failed to marshal authenticate reply: "+ + "%v", err.Error()) continue } c.SendMessage(reply, nil) continue } + + // Check if the client is using limited RPC credentials and + // error when not authorized to call the supplied RPC. + if !c.isAdmin { + if _, ok := rpcLimited[req.Method]; !ok { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidParams.Code, + Message: "limited user not authorized for this method", + } + // Marshal and send response. + reply, err = createMarshalledReply("", req.ID, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal parse failure "+ + "reply: %v", err) + continue + } + c.SendMessage(reply, nil) + continue + } + } + + // Asynchronously handle the request. A semaphore is used to + // limit the number of concurrent requests currently being + // serviced. If the semaphore can not be acquired, simply wait + // until a request finished before reading the next RPC request + // from the websocket client. + // + // This could be a little fancier by timing out and erroring + // when it takes too long to service the request, but if that is + // done, the read of the next request should not be blocked by + // this semaphore, otherwise the next request will be read and + // will probably sit here for another few seconds before timing + // out as well. This will cause the total timeout duration for + // later requests to be much longer than the check here would + // imply. + // + // If a timeout is added, the semaphore acquiring should be + // moved inside of the new goroutine with a select statement + // that also reads a time.After channel. This will unblock the + // read of the next request from the websocket client and allow + // many requests to be waited on concurrently. + c.serviceRequestSem.acquire() + go func() { + c.serviceRequest(cmd) + c.serviceRequestSem.release() + }() } - // Asynchronously handle the request. A semaphore is used to - // limit the number of concurrent requests currently being - // serviced. If the semaphore can not be acquired, simply wait - // until a request finished before reading the next RPC request - // from the websocket client. - // - // This could be a little fancier by timing out and erroring - // when it takes too long to service the request, but if that is - // done, the read of the next request should not be blocked by - // this semaphore, otherwise the next request will be read and - // will probably sit here for another few seconds before timing - // out as well. This will cause the total timeout duration for - // later requests to be much longer than the check here would - // imply. - // - // If a timeout is added, the semaphore acquiring should be - // moved inside of the new goroutine with a select statement - // that also reads a time.After channel. This will unblock the - // read of the next request from the websocket client and allow - // many requests to be waited on concurrently. - c.serviceRequestSem.acquire() - go func() { - c.serviceRequest(cmd) + // Process a batched request + if batchedRequest { + var batchedRequests []interface{} + var results []json.RawMessage + var batchSize int + var reply json.RawMessage + c.serviceRequestSem.acquire() + err = json.Unmarshal(msg, &batchedRequests) + if err != nil { + // Only process requests from authenticated clients + if !c.authenticated { + break out + } + + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCParse.Code, + Message: fmt.Sprintf("Failed to parse request: %v", + err), + } + reply, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + } + + if reply != nil { + results = append(results, reply) + } + } + + if err == nil { + // Response with an empty batch error if the batch size is zero + if len(batchedRequests) == 0 { + if !c.authenticated { + break out + } + + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: empty batch", + } + reply, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + } + + if reply != nil { + results = append(results, reply) + } + } + + // Process each batch entry individually + if len(batchedRequests) > 0 { + batchSize = len(batchedRequests) + for _, entry := range batchedRequests { + var reqBytes []byte + reqBytes, err = json.Marshal(entry) + if err != nil { + // Only process requests from authenticated clients + if !c.authenticated { + break out + } + + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: fmt.Sprintf("Invalid request: %v", + err), + } + reply, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + + var req btcjson.Request + err := json.Unmarshal(reqBytes, &req) + if err != nil { + // Only process requests from authenticated clients + if !c.authenticated { + break out + } + + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: fmt.Sprintf("Invalid request: %v", + err), + } + reply, err = btcjson.MarshalResponse(btcjson.RpcVersion2, nil, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to create reply: %v", err) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + + if req.Method == "" || req.Params == nil { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidRequest.Code, + Message: "Invalid request: malformed", + } + reply, err := createMarshalledReply(req.Jsonrpc, req.ID, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + + // Valid requests with no ID (notifications) must not have a response + // per the JSON-RPC spec. + if req.ID == nil { + if !c.authenticated { + break out + } + continue + } + + cmd := parseCmd(&req) + if cmd.err != nil { + // Only process requests from authenticated clients + if !c.authenticated { + break out + } + + reply, err = createMarshalledReply(cmd.jsonrpc, cmd.id, nil, cmd.err) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply: %v", err) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + + rpcsLog.Debugf("Received command <%s> from %s", cmd.method, c.addr) + + // Check auth. The client is immediately disconnected if the + // first request of an unauthentiated websocket client is not + // the authenticate request, an authenticate request is received + // when the client is already authenticated, or incorrect + // authentication credentials are provided in the request. + switch authCmd, ok := cmd.cmd.(*btcjson.AuthenticateCmd); { + case c.authenticated && ok: + rpcsLog.Warnf("Websocket client %s is already authenticated", + c.addr) + break out + case !c.authenticated && !ok: + rpcsLog.Warnf("Unauthenticated websocket message " + + "received") + break out + case !c.authenticated: + // Check credentials. + login := authCmd.Username + ":" + authCmd.Passphrase + auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(login)) + authSha := sha256.Sum256([]byte(auth)) + cmp := subtle.ConstantTimeCompare(authSha[:], c.server.authsha[:]) + limitcmp := subtle.ConstantTimeCompare(authSha[:], c.server.limitauthsha[:]) + if cmp != 1 && limitcmp != 1 { + rpcsLog.Warnf("Auth failure.") + break out + } + + c.authenticated = true + c.isAdmin = cmp == 1 + + // Marshal and send response. + reply, err = createMarshalledReply(cmd.jsonrpc, cmd.id, nil, nil) + if err != nil { + rpcsLog.Errorf("Failed to marshal authenticate reply: "+ + "%v", err.Error()) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + + // Check if the client is using limited RPC credentials and + // error when not authorized to call the supplied RPC. + if !c.isAdmin { + if _, ok := rpcLimited[req.Method]; !ok { + jsonErr := &btcjson.RPCError{ + Code: btcjson.ErrRPCInvalidParams.Code, + Message: "limited user not authorized for this method", + } + // Marshal and send response. + reply, err = createMarshalledReply(req.Jsonrpc, req.ID, nil, jsonErr) + if err != nil { + rpcsLog.Errorf("Failed to marshal parse failure "+ + "reply: %v", err) + continue + } + + if reply != nil { + results = append(results, reply) + } + continue + } + } + + // Lookup the websocket extension for the command, if it doesn't + // exist fallback to handling the command as a standard command. + var resp interface{} + wsHandler, ok := wsHandlers[cmd.method] + if ok { + resp, err = wsHandler(c, cmd.cmd) + } else { + resp, err = c.server.standardCmdResult(cmd, nil) + } + + // Marshal request output. + reply, err := createMarshalledReply(cmd.jsonrpc, cmd.id, resp, err) + if err != nil { + rpcsLog.Errorf("Failed to marshal reply for <%s> "+ + "command: %v", cmd.method, err) + return + } + + if reply != nil { + results = append(results, reply) + } + } + } + } + + // generate reply + var payload = []byte{} + if batchedRequest && batchSize > 0 { + if len(results) > 0 { + // Form the batched response json + var buffer bytes.Buffer + buffer.WriteByte('[') + for idx, marshalledReply := range results { + if idx == len(results)-1 { + buffer.Write(marshalledReply) + buffer.WriteByte(']') + break + } + buffer.Write(marshalledReply) + buffer.WriteByte(',') + } + payload = buffer.Bytes() + } + } + + if !batchedRequest || batchSize == 0 { + // Respond with the first results entry for single requests + if len(results) > 0 { + payload = results[0] + } + } + + c.SendMessage(payload, nil) c.serviceRequestSem.release() - }() + } } // Ensure the connection is closed. @@ -1495,7 +1777,7 @@ func (c *wsClient) serviceRequest(r *parsedRPCCmd) { } else { result, err = c.server.standardCmdResult(r, nil) } - reply, err := createMarshalledReply(r.id, result, err) + reply, err := createMarshalledReply(r.jsonrpc, r.id, result, err) if err != nil { rpcsLog.Errorf("Failed to marshal reply for <%s> "+ "command: %v", r.method, err) @@ -2125,7 +2407,7 @@ func rescanBlock(wsc *wsClient, lookups *rescanKeys, blk *btcutil.Block) { ntfn := btcjson.NewRecvTxNtfn(txHex, blockDetails(blk, tx.Index())) - marshalledJSON, err := btcjson.MarshalCmd(nil, ntfn) + marshalledJSON, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, ntfn) if err != nil { rpcsLog.Errorf("Failed to marshal recvtx notification: %v", err) return @@ -2492,7 +2774,7 @@ fetchRange: hashList[i].String(), blk.Height(), blk.MsgBlock().Header.Timestamp.Unix(), ) - mn, err := btcjson.MarshalCmd(nil, n) + mn, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, n) if err != nil { rpcsLog.Errorf("Failed to marshal rescan "+ "progress notification: %v", err) @@ -2637,7 +2919,7 @@ func handleRescan(wsc *wsClient, icmd interface{}) (interface{}, error) { lastBlockHash.String(), lastBlock.Height(), lastBlock.MsgBlock().Header.Timestamp.Unix(), ) - if mn, err := btcjson.MarshalCmd(nil, n); err != nil { + if mn, err := btcjson.MarshalCmd(btcjson.RpcVersion1, nil, n); err != nil { rpcsLog.Errorf("Failed to marshal rescan finished "+ "notification: %v", err) } else { diff --git a/sample-btcd.conf b/sample-btcd.conf index e2beb99cf8..0a765fcabe 100644 --- a/sample-btcd.conf +++ b/sample-btcd.conf @@ -324,7 +324,7 @@ ; sizes have the highest priority. One consequence of this is that as low-fee ; or free transactions age, they raise in priority thereby making them more ; likely to be included in this section of a new block. This value is limited -; by the blackmaxsize option and will be limited as needed. +; by the blockmaxsize option and will be limited as needed. ; blockprioritysize=50000 diff --git a/server.go b/server.go index c9f23fa638..ba7932a1a4 100644 --- a/server.go +++ b/server.go @@ -1321,8 +1321,12 @@ func (sp *serverPeer) OnNotFound(p *peer.Peer, msg *wire.MsgNotFound) { switch inv.Type { case wire.InvTypeBlock: numBlocks++ + case wire.InvTypeWitnessBlock: + numBlocks++ case wire.InvTypeTx: numTxns++ + case wire.InvTypeWitnessTx: + numTxns++ default: peerLog.Debugf("Invalid inv type '%d' in notfound message from %s", inv.Type, sp) diff --git a/txscript/README.md b/txscript/README.md index 5793173451..004c586d61 100644 --- a/txscript/README.md +++ b/txscript/README.md @@ -1,9 +1,9 @@ txscript ======== -[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://godoc.org/github.com/btcsuite/btcd/txscript?status.png)](http://godoc.org/github.com/btcsuite/btcd/txscript) +[![GoDoc](https://pkg.go.dev/github.com/btcsuite/btcd/txscript?status.png)](https://pkg.go.dev/github.com/btcsuite/btcd/txscript) Package txscript implements the bitcoin transaction script language. There is a comprehensive test suite. @@ -26,15 +26,15 @@ $ go get -u github.com/btcsuite/btcd/txscript ## Examples -* [Standard Pay-to-pubkey-hash Script](http://godoc.org/github.com/btcsuite/btcd/txscript#example-PayToAddrScript) +* [Standard Pay-to-pubkey-hash Script](https://pkg.go.dev/github.com/btcsuite/btcd/txscript#example-PayToAddrScript) Demonstrates creating a script which pays to a bitcoin address. It also prints the created script hex and uses the DisasmString function to display the disassembled script. -* [Extracting Details from Standard Scripts](http://godoc.org/github.com/btcsuite/btcd/txscript#example-ExtractPkScriptAddrs) +* [Extracting Details from Standard Scripts](https://pkg.go.dev/github.com/btcsuite/btcd/txscript#example-ExtractPkScriptAddrs) Demonstrates extracting information from a standard public key script. -* [Manually Signing a Transaction Output](http://godoc.org/github.com/btcsuite/btcd/txscript#example-SignTxOutput) +* [Manually Signing a Transaction Output](https://pkg.go.dev/github.com/btcsuite/btcd/txscript#example-SignTxOutput) Demonstrates manually creating and signing a redeem transaction. ## GPG Verification Key diff --git a/txscript/hashcache_test.go b/txscript/hashcache_test.go index 389918e2f2..cee59b9956 100644 --- a/txscript/hashcache_test.go +++ b/txscript/hashcache_test.go @@ -13,12 +13,16 @@ import ( "github.com/davecgh/go-spew/spew" ) +func init() { + rand.Seed(time.Now().Unix()) +} + // genTestTx creates a random transaction for uses within test cases. func genTestTx() (*wire.MsgTx, error) { tx := wire.NewMsgTx(2) tx.Version = rand.Int31() - numTxins := rand.Intn(11) + numTxins := 1 + rand.Intn(11) for i := 0; i < numTxins; i++ { randTxIn := wire.TxIn{ PreviousOutPoint: wire.OutPoint{ @@ -34,7 +38,7 @@ func genTestTx() (*wire.MsgTx, error) { tx.TxIn = append(tx.TxIn, &randTxIn) } - numTxouts := rand.Intn(11) + numTxouts := 1 + rand.Intn(11) for i := 0; i < numTxouts; i++ { randTxOut := wire.TxOut{ Value: rand.Int63(), @@ -56,8 +60,6 @@ func genTestTx() (*wire.MsgTx, error) { func TestHashCacheAddContainsHashes(t *testing.T) { t.Parallel() - rand.Seed(time.Now().Unix()) - cache := NewHashCache(10) var err error @@ -109,8 +111,6 @@ func TestHashCacheAddContainsHashes(t *testing.T) { func TestHashCacheAddGet(t *testing.T) { t.Parallel() - rand.Seed(time.Now().Unix()) - cache := NewHashCache(10) // To start, we'll generate a random transaction and compute the set of @@ -144,8 +144,6 @@ func TestHashCacheAddGet(t *testing.T) { func TestHashCachePurge(t *testing.T) { t.Parallel() - rand.Seed(time.Now().Unix()) - cache := NewHashCache(10) var err error diff --git a/txscript/opcode.go b/txscript/opcode.go index 5ffb398277..a878a9667b 100644 --- a/txscript/opcode.go +++ b/txscript/opcode.go @@ -656,6 +656,79 @@ func (pop *parsedOpcode) isDisabled() bool { } } +// checkParseableInScript checks whether or not the current opcode is able to be +// parsed at a certain position in a script. +// This returns the position of the next opcode to be parsed in the script. +func (pop *parsedOpcode) checkParseableInScript(script []byte, scriptPos int) (int, error) { + // Parse data out of instruction. + switch { + // No additional data. Note that some of the opcodes, notably + // OP_1NEGATE, OP_0, and OP_[1-16] represent the data + // themselves. + case pop.opcode.length == 1: + scriptPos++ + + // Data pushes of specific lengths -- OP_DATA_[1-75]. + case pop.opcode.length > 1: + if len(script[scriptPos:]) < pop.opcode.length { + str := fmt.Sprintf("opcode %s requires %d "+ + "bytes, but script only has %d remaining", + pop.opcode.name, pop.opcode.length, len(script[scriptPos:])) + return 0, scriptError(ErrMalformedPush, str) + } + + // Slice out the data. + pop.data = script[scriptPos+1 : scriptPos+pop.opcode.length] + scriptPos += pop.opcode.length + + // Data pushes with parsed lengths -- OP_PUSHDATAP{1,2,4}. + case pop.opcode.length < 0: + var l uint + off := scriptPos + 1 + + if len(script[off:]) < -pop.opcode.length { + str := fmt.Sprintf("opcode %s requires %d "+ + "bytes, but script only has %d remaining", + pop.opcode.name, -pop.opcode.length, len(script[off:])) + return 0, scriptError(ErrMalformedPush, str) + } + + // Next -length bytes are little endian length of data. + switch pop.opcode.length { + case -1: + l = uint(script[off]) + case -2: + l = ((uint(script[off+1]) << 8) | + uint(script[off])) + case -4: + l = ((uint(script[off+3]) << 24) | + (uint(script[off+2]) << 16) | + (uint(script[off+1]) << 8) | + uint(script[off])) + default: + str := fmt.Sprintf("invalid opcode length %d", + pop.opcode.length) + return 0, scriptError(ErrMalformedPush, str) + } + + // Move offset to beginning of the data. + off += -pop.opcode.length + + // Disallow entries that do not fit script or were + // sign extended. + if int(l) > len(script[off:]) || int(l) < 0 { + str := fmt.Sprintf("opcode %s pushes %d bytes, "+ + "but script only has %d remaining", + pop.opcode.name, int(l), len(script[off:])) + return 0, scriptError(ErrMalformedPush, str) + } + + pop.data = script[off : off+int(l)] + scriptPos += 1 - pop.opcode.length + int(l) + } + return scriptPos, nil +} + // alwaysIllegal returns whether or not the opcode is always illegal when passed // over by the program counter even if in a non-executed branch (it isn't a // coincidence that they are conditionals). diff --git a/txscript/script.go b/txscript/script.go index aac3d4aaaa..92a50e3761 100644 --- a/txscript/script.go +++ b/txscript/script.go @@ -196,86 +196,58 @@ func IsPushOnlyScript(script []byte) bool { // the list of parsed opcodes up to the point of failure along with the error. func parseScriptTemplate(script []byte, opcodes *[256]opcode) ([]parsedOpcode, error) { retScript := make([]parsedOpcode, 0, len(script)) + var err error for i := 0; i < len(script); { instr := script[i] op := &opcodes[instr] pop := parsedOpcode{opcode: op} + i, err = pop.checkParseableInScript(script, i) + if err != nil { + return retScript, err + } - // Parse data out of instruction. - switch { - // No additional data. Note that some of the opcodes, notably - // OP_1NEGATE, OP_0, and OP_[1-16] represent the data - // themselves. - case op.length == 1: - i++ - - // Data pushes of specific lengths -- OP_DATA_[1-75]. - case op.length > 1: - if len(script[i:]) < op.length { - str := fmt.Sprintf("opcode %s requires %d "+ - "bytes, but script only has %d remaining", - op.name, op.length, len(script[i:])) - return retScript, scriptError(ErrMalformedPush, - str) - } + retScript = append(retScript, pop) + } - // Slice out the data. - pop.data = script[i+1 : i+op.length] - i += op.length - - // Data pushes with parsed lengths -- OP_PUSHDATAP{1,2,4}. - case op.length < 0: - var l uint - off := i + 1 - - if len(script[off:]) < -op.length { - str := fmt.Sprintf("opcode %s requires %d "+ - "bytes, but script only has %d remaining", - op.name, -op.length, len(script[off:])) - return retScript, scriptError(ErrMalformedPush, - str) - } + return retScript, nil +} - // Next -length bytes are little endian length of data. - switch op.length { - case -1: - l = uint(script[off]) - case -2: - l = ((uint(script[off+1]) << 8) | - uint(script[off])) - case -4: - l = ((uint(script[off+3]) << 24) | - (uint(script[off+2]) << 16) | - (uint(script[off+1]) << 8) | - uint(script[off])) - default: - str := fmt.Sprintf("invalid opcode length %d", - op.length) - return retScript, scriptError(ErrMalformedPush, - str) - } +// checkScriptTemplateParseable is the same as parseScriptTemplate but does not +// return the list of opcodes up until the point of failure so that this can be +// used in functions which do not necessarily have a need for the failed list of +// opcodes, such as IsUnspendable. +// +// This function returns a pointer to a byte. This byte is nil if the parsing +// has an error, or if the script length is zero. If the script length is not +// zero and parsing succeeds, then the first opcode parsed will be returned. +// +// Not returning the full opcode list up until failure also has the benefit of +// reducing GC pressure, as the list would get immediately thrown away. +func checkScriptTemplateParseable(script []byte, opcodes *[256]opcode) (*byte, error) { + var err error - // Move offset to beginning of the data. - off += -op.length - - // Disallow entries that do not fit script or were - // sign extended. - if int(l) > len(script[off:]) || int(l) < 0 { - str := fmt.Sprintf("opcode %s pushes %d bytes, "+ - "but script only has %d remaining", - op.name, int(l), len(script[off:])) - return retScript, scriptError(ErrMalformedPush, - str) - } + // A script of length zero is an unspendable script but it is parseable. + var firstOpcode byte + var numParsedInstr uint = 0 - pop.data = script[off : off+int(l)] - i += 1 - op.length + int(l) + for i := 0; i < len(script); { + instr := script[i] + op := &opcodes[instr] + pop := parsedOpcode{opcode: op} + i, err = pop.checkParseableInScript(script, i) + if err != nil { + return nil, err } - retScript = append(retScript, pop) + // if this is a op_return then it is unspendable so we set the first + // parsed instruction in case it's an op_return + if numParsedInstr == 0 { + firstOpcode = pop.opcode.value + } + numParsedInstr++ } - return retScript, nil + return &firstOpcode, nil } // parseScript preparses the script in bytes into a list of parsedOpcodes while @@ -851,10 +823,14 @@ func getWitnessSigOps(pkScript []byte, witness wire.TxWitness) int { // guaranteed to fail at execution. This allows inputs to be pruned instantly // when entering the UTXO set. func IsUnspendable(pkScript []byte) bool { - pops, err := parseScript(pkScript) + // Not provably unspendable + if len(pkScript) == 0 { + return false + } + firstOpcode, err := checkScriptTemplateParseable(pkScript, &opcodeArray) if err != nil { return true } - return len(pops) > 0 && pops[0].opcode.value == OP_RETURN + return firstOpcode != nil && *firstOpcode == OP_RETURN } diff --git a/txscript/script_test.go b/txscript/script_test.go index 6a725e275c..34c8ef9740 100644 --- a/txscript/script_test.go +++ b/txscript/script_test.go @@ -4301,6 +4301,28 @@ func TestIsUnspendable(t *testing.T) { 0xfa, 0x0b, 0x5c, 0x88, 0xac}, expected: false, }, + { + // Spendable + pkScript: []byte{0xa9, 0x14, 0x82, 0x1d, 0xba, 0x94, 0xbc, 0xfb, + 0xa2, 0x57, 0x36, 0xa3, 0x9e, 0x5d, 0x14, 0x5d, 0x69, 0x75, + 0xba, 0x8c, 0x0b, 0x42, 0x87}, + expected: false, + }, + { + // Not Necessarily Unspendable + pkScript: []byte{}, + expected: false, + }, + { + // Spendable + pkScript: []byte{OP_TRUE}, + expected: false, + }, + { + // Unspendable + pkScript: []byte{OP_RETURN}, + expected: true, + }, } for i, test := range tests { @@ -4312,3 +4334,16 @@ func TestIsUnspendable(t *testing.T) { } } } + +// BenchmarkIsUnspendable adds a benchmark to compare the time and allocations +// necessary for the IsUnspendable function. +func BenchmarkIsUnspendable(b *testing.B) { + pkScriptToUse := []byte{0xa9, 0x14, 0x82, 0x1d, 0xba, 0x94, 0xbc, 0xfb, 0xa2, 0x57, 0x36, 0xa3, 0x9e, 0x5d, 0x14, 0x5d, 0x69, 0x75, 0xba, 0x8c, 0x0b, 0x42, 0x87} + var res bool = false + for i := 0; i < b.N; i++ { + res = IsUnspendable(pkScriptToUse) + } + if res { + b.Fatalf("Benchmark should never have res be %t\n", res) + } +} diff --git a/txscript/standard.go b/txscript/standard.go index a7e929d101..2cad218e95 100644 --- a/txscript/standard.go +++ b/txscript/standard.go @@ -1,4 +1,4 @@ -// Copyright (c) 2013-2017 The btcsuite developers +// Copyright (c) 2013-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. @@ -58,6 +58,7 @@ const ( WitnessV0ScriptHashTy // Pay to witness script hash. MultiSigTy // Multi signature. NullDataTy // Empty data-only (provably prunable). + WitnessUnknownTy // Witness unknown ) // scriptClassToName houses the human-readable strings which describe each @@ -71,6 +72,7 @@ var scriptClassToName = []string{ WitnessV0ScriptHashTy: "witness_v0_scripthash", MultiSigTy: "multisig", NullDataTy: "nulldata", + WitnessUnknownTy: "witness_unknown", } // String implements the Stringer interface by returning the name of @@ -188,6 +190,22 @@ func GetScriptClass(script []byte) ScriptClass { return typeOfScript(pops) } +// NewScriptClass returns the ScriptClass corresponding to the string name +// provided as argument. ErrUnsupportedScriptType error is returned if the +// name doesn't correspond to any known ScriptClass. +// +// Not to be confused with GetScriptClass. +func NewScriptClass(name string) (*ScriptClass, error) { + for i, n := range scriptClassToName { + if n == name { + value := ScriptClass(i) + return &value, nil + } + } + + return nil, fmt.Errorf("%w: %s", ErrUnsupportedScriptType, name) +} + // expectedInputs returns the number of arguments required by a script. // If the script is of unknown type such that the number can not be determined // then -1 is returned. We are an internal function and thus assume that class diff --git a/txscript/standard_test.go b/txscript/standard_test.go index e24d5f615b..37dd8f8a37 100644 --- a/txscript/standard_test.go +++ b/txscript/standard_test.go @@ -1,4 +1,4 @@ -// Copyright (c) 2013-2017 The btcsuite developers +// Copyright (c) 2013-2020 The btcsuite developers // Use of this source code is governed by an ISC // license that can be found in the LICENSE file. @@ -7,6 +7,7 @@ package txscript import ( "bytes" "encoding/hex" + "errors" "reflect" "testing" @@ -1213,3 +1214,40 @@ func TestNullDataScript(t *testing.T) { } } } + +// TestNewScriptClass tests whether NewScriptClass returns a valid ScriptClass. +func TestNewScriptClass(t *testing.T) { + tests := []struct { + name string + scriptName string + want *ScriptClass + wantErr error + }{ + { + name: "NewScriptClass - ok", + scriptName: NullDataTy.String(), + want: func() *ScriptClass { + s := NullDataTy + return &s + }(), + }, + { + name: "NewScriptClass - invalid", + scriptName: "foo", + wantErr: ErrUnsupportedScriptType, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := NewScriptClass(tt.scriptName) + if err != nil && !errors.Is(err, tt.wantErr) { + t.Errorf("NewScriptClass() error = %v, wantErr %v", err, tt.wantErr) + return + } + + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("NewScriptClass() got = %v, want %v", got, tt.want) + } + }) + } +} diff --git a/upnp.go b/upnp.go index feb256d072..c74e4ed79a 100644 --- a/upnp.go +++ b/upnp.go @@ -36,6 +36,7 @@ import ( "bytes" "encoding/xml" "errors" + "fmt" "net" "net/http" "os" @@ -229,7 +230,7 @@ func getServiceURL(rootURL string) (url string, err error) { } defer r.Body.Close() if r.StatusCode >= 400 { - err = errors.New(string(r.StatusCode)) + err = errors.New(fmt.Sprint(r.StatusCode)) return } var root root diff --git a/version.go b/version.go index ac294de232..d6ff9171aa 100644 --- a/version.go +++ b/version.go @@ -17,7 +17,7 @@ const semanticAlphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqr // versioning 2.0.0 spec (http://semver.org/). const ( appMajor uint = 0 - appMinor uint = 21 + appMinor uint = 22 appPatch uint = 0 // appPreRelease MUST only contain characters from semanticAlphabet diff --git a/wire/README.md b/wire/README.md index 646bc0a309..8660bbfd54 100644 --- a/wire/README.md +++ b/wire/README.md @@ -1,9 +1,9 @@ wire ==== -[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)](https://travis-ci.org/btcsuite/btcd) +[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions) [![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org) -[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd/wire) +[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/btcsuite/btcd/wire) ======= Package wire implements the bitcoin wire protocol. A comprehensive suite of diff --git a/wire/fixedIO_test.go b/wire/fixedIO_test.go index 731c463892..ccd67ae411 100644 --- a/wire/fixedIO_test.go +++ b/wire/fixedIO_test.go @@ -68,7 +68,7 @@ func (fr *fixedReader) Read(p []byte) (n int, err error) { func newFixedReader(max int, buf []byte) io.Reader { b := make([]byte, max) if buf != nil { - copy(b[:], buf) + copy(b, buf) } iobuf := bytes.NewBuffer(b) diff --git a/wire/message.go b/wire/message.go index e937647702..6d3147a81d 100644 --- a/wire/message.go +++ b/wire/message.go @@ -57,6 +57,7 @@ const ( CmdCFilter = "cfilter" CmdCFHeaders = "cfheaders" CmdCFCheckpt = "cfcheckpt" + CmdSendAddrV2 = "sendaddrv2" ) // MessageEncoding represents the wire message encoding format to be used. @@ -99,6 +100,9 @@ func makeEmptyMessage(command string) (Message, error) { case CmdVerAck: msg = &MsgVerAck{} + case CmdSendAddrV2: + msg = &MsgSendAddrV2{} + case CmdGetAddr: msg = &MsgGetAddr{} @@ -213,7 +217,7 @@ func readMessageHeader(r io.Reader) (int, *messageHeader, error) { readElements(hr, &hdr.magic, &command, &hdr.length, &hdr.checksum) // Strip trailing zeros from command string. - hdr.command = string(bytes.TrimRight(command[:], string(0))) + hdr.command = string(bytes.TrimRight(command[:], "\x00")) return n, &hdr, nil } @@ -401,7 +405,7 @@ func ReadMessageWithEncodingN(r io.Reader, pver uint32, btcnet BitcoinNet, // Test checksum. checksum := chainhash.DoubleHashB(payload)[0:4] - if !bytes.Equal(checksum[:], hdr.checksum[:]) { + if !bytes.Equal(checksum, hdr.checksum[:]) { str := fmt.Sprintf("payload checksum failed - header "+ "indicates %v, but actual checksum is %v.", hdr.checksum, checksum) diff --git a/wire/msgsendaddrv2.go b/wire/msgsendaddrv2.go new file mode 100644 index 0000000000..d6d19efb27 --- /dev/null +++ b/wire/msgsendaddrv2.go @@ -0,0 +1,42 @@ +package wire + +import ( + "io" +) + +// MsgSendAddrV2 defines a bitcoin sendaddrv2 message which is used for a peer +// to signal support for receiving ADDRV2 messages (BIP155). It implements the +// Message interface. +// +// This message has no payload. +type MsgSendAddrV2 struct{} + +// BtcDecode decodes r using the bitcoin protocol encoding into the receiver. +// This is part of the Message interface implementation. +func (msg *MsgSendAddrV2) BtcDecode(r io.Reader, pver uint32, enc MessageEncoding) error { + return nil +} + +// BtcEncode encodes the receiver to w using the bitcoin protocol encoding. +// This is part of the Message interface implementation. +func (msg *MsgSendAddrV2) BtcEncode(w io.Writer, pver uint32, enc MessageEncoding) error { + return nil +} + +// Command returns the protocol command string for the message. This is part +// of the Message interface implementation. +func (msg *MsgSendAddrV2) Command() string { + return CmdSendAddrV2 +} + +// MaxPayloadLength returns the maximum length the payload can be for the +// receiver. This is part of the Message interface implementation. +func (msg *MsgSendAddrV2) MaxPayloadLength(pver uint32) uint32 { + return 0 +} + +// NewMsgSendAddrV2 returns a new bitcoin sendaddrv2 message that conforms to the +// Message interface. +func NewMsgSendAddrV2() *MsgSendAddrV2 { + return &MsgSendAddrV2{} +} diff --git a/wire/msgtx.go b/wire/msgtx.go index 2bf27958e2..50459c8fec 100644 --- a/wire/msgtx.go +++ b/wire/msgtx.go @@ -109,14 +109,38 @@ const ( maxWitnessItemSize = 11000 ) -// witnessMarkerBytes are a pair of bytes specific to the witness encoding. If -// this sequence is encoutered, then it indicates a transaction has iwtness -// data. The first byte is an always 0x00 marker byte, which allows decoders to -// distinguish a serialized transaction with witnesses from a regular (legacy) -// one. The second byte is the Flag field, which at the moment is always 0x01, -// but may be extended in the future to accommodate auxiliary non-committed -// fields. -var witessMarkerBytes = []byte{0x00, 0x01} +// TxFlagMarker is the first byte of the FLAG field in a bitcoin tx +// message. It allows decoders to distinguish a regular serialized +// transaction from one that would require a different parsing logic. +// +// Position of FLAG in a bitcoin tx message: +// ┌─────────┬────────────────────┬─────────────┬─────┐ +// │ VERSION │ FLAG │ TX-IN-COUNT │ ... │ +// │ 4 bytes │ 2 bytes (optional) │ varint │ │ +// └─────────┴────────────────────┴─────────────┴─────┘ +// +// Zooming into the FLAG field: +// ┌── FLAG ─────────────┬────────┐ +// │ TxFlagMarker (0x00) │ TxFlag │ +// │ 1 byte │ 1 byte │ +// └─────────────────────┴────────┘ +const TxFlagMarker = 0x00 + +// TxFlag is the second byte of the FLAG field in a bitcoin tx message. +// It indicates the decoding logic to use in the transaction parser, if +// TxFlagMarker is detected in the tx message. +// +// As of writing this, only the witness flag (0x01) is supported, but may be +// extended in the future to accommodate auxiliary non-committed fields. +type TxFlag = byte + +const ( + // WitnessFlag is a flag specific to witness encoding. If the TxFlagMarker + // is encountered followed by the WitnessFlag, then it indicates a + // transaction has witness data. This allows decoders to distinguish a + // serialized transaction with witnesses from a legacy one. + WitnessFlag TxFlag = 0x01 +) // scriptFreeList defines a free list of byte slices (up to the maximum number // defined by the freeListMaxItems constant) that have a cap according to the @@ -428,18 +452,19 @@ func (msg *MsgTx) BtcDecode(r io.Reader, pver uint32, enc MessageEncoding) error return err } - // A count of zero (meaning no TxIn's to the uninitiated) indicates - // this is a transaction with witness data. - var flag [1]byte - if count == 0 && enc == WitnessEncoding && !(version >= 3 && txType == 6) { - // Next, we need to read the flag, which is a single byte. + // A count of zero (meaning no TxIn's to the uninitiated) means that the + // value is a TxFlagMarker, and hence indicates the presence of a flag. + var flag [1]TxFlag + if count == TxFlagMarker && enc == WitnessEncoding && !(version >= 3 && txType == 6) { + // The count varint was in fact the flag marker byte. Next, we need to + // read the flag value, which is a single byte. if _, err = io.ReadFull(r, flag[:]); err != nil { return err } - // At the moment, the flag MUST be 0x01. In the future other - // flag types may be supported. - if flag[0] != 0x01 { + // At the moment, the flag MUST be WitnessFlag (0x01). In the future + // other flag types may be supported. + if flag[0] != WitnessFlag { str := fmt.Sprintf("witness tx but flag byte is %x", flag) return messageError("MsgTx.BtcDecode", str) } @@ -714,14 +739,11 @@ func (msg *MsgTx) BtcEncode(w io.Writer, pver uint32, enc MessageEncoding) error // defined in BIP0144. doWitness := enc == WitnessEncoding && msg.HasWitness() if doWitness { - // After the txn's Version field, we include two additional - // bytes specific to the witness encoding. The first byte is an - // always 0x00 marker byte, which allows decoders to - // distinguish a serialized transaction with witnesses from a - // regular (legacy) one. The second byte is the Flag field, - // which at the moment is always 0x01, but may be extended in - // the future to accommodate auxiliary non-committed fields. - if _, err := w.Write(witessMarkerBytes); err != nil { + // After the transaction's Version field, we include two additional + // bytes specific to the witness encoding. This byte sequence is known + // as a flag. The first byte is a marker byte (TxFlagMarker) and the + // second one is the flag value to indicate presence of witness data. + if _, err := w.Write([]byte{TxFlagMarker, WitnessFlag}); err != nil { return err } } diff --git a/wire/msgtx_test.go b/wire/msgtx_test.go index dd809f81e2..66965043e6 100644 --- a/wire/msgtx_test.go +++ b/wire/msgtx_test.go @@ -935,9 +935,9 @@ var multiWitnessTx = &MsgTx{ // tests. var multiWitnessTxEncoded = []byte{ 0x1, 0x0, 0x0, 0x0, // Version - 0x0, // Marker byte indicating 0 inputs, or a segwit encoded tx - 0x1, // Flag byte - 0x1, // Varint for number of inputs + TxFlagMarker, // Marker byte indicating 0 inputs, or a segwit encoded tx + WitnessFlag, // Flag byte + 0x1, // Varint for number of inputs 0xa5, 0x33, 0x52, 0xd5, 0x13, 0x57, 0x66, 0xf0, 0x30, 0x76, 0x59, 0x74, 0x18, 0x26, 0x3d, 0xa2, 0xd9, 0xc9, 0x58, 0x31, 0x59, 0x68, 0xfe, 0xa8, @@ -978,9 +978,9 @@ var multiWitnessTxEncoded = []byte{ // being set to 0x01, the flag is 0x00, which should trigger a decoding error. var multiWitnessTxEncodedNonZeroFlag = []byte{ 0x1, 0x0, 0x0, 0x0, // Version - 0x0, // Marker byte indicating 0 inputs, or a segwit encoded tx - 0x0, // Incorrect flag byte (should be 0x01) - 0x1, // Varint for number of inputs + TxFlagMarker, // Marker byte indicating 0 inputs, or a segwit encoded tx + 0x0, // Incorrect flag byte (should be 0x01) + 0x1, // Varint for number of inputs 0xa5, 0x33, 0x52, 0xd5, 0x13, 0x57, 0x66, 0xf0, 0x30, 0x76, 0x59, 0x74, 0x18, 0x26, 0x3d, 0xa2, 0xd9, 0xc9, 0x58, 0x31, 0x59, 0x68, 0xfe, 0xa8,