feat!: merges feature-dan into development #4913
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chore: merge development into feature-dan
…t#4470) Description --- - adds CodeTemplateRegistration output type - adds TemplateRegistration to SideChainFeatures - add grpc methods for registering a code template Motivation and Context --- Code templates need to be committed to on L1. The `binary_url` and `repo_url` fields are not validated by base node consensus because: 1. The rust implementation of Multiaddr does not currently support paths in http addresses e.g. `/dns4/github.com/tcp/443/http/tari-project/tari.git` is not well-formed however [the spec](https://multiformats.io/multiaddr/) says it should be. 2. Url parsing is complex and adding it to consensus code could introduce bugs (general/security/DoS) Depends on tari-project#4466 Depends on tari-project#4496 How Has This Been Tested? --- TODO: add cucumber tests for this output type
Description --- Add GRPC call for getting shard key for a public key.
* development: (72 commits) fix: reinsert transactions from failed block (tari-project#4675) fix: stray clippy error (tari-project#4685) fix(wallet): mark mined_height as null when pending outputs are cancelled (tari-project#4686) chore: updated dependancies (tari-project#4684) fix(p2p): remove DETACH flag usage (tari-project#4682) fix(comms): simplify and remove possibility of deadlock from pipelines and substream close (tari-project#4676) feat(ci): add default CI and FFI testing with custom dispatch (tari-project#4672) chore: remove broken test (tari-project#4678) fix: fix potential race condition between add_block and sync (tari-project#4677) fix deadlock (tari-project#4674) fix: add burn funds command to console wallet (see issue tari-project#4547) (tari-project#4655) v0.38.3 fix: fee estimate (tari-project#4656) fix(comms/messaging): fix possible deadlock in outbound pipeline (tari-project#4657) fix: replace Luhn checksum with DammSum (tari-project#4639) fix(core/sync): handle deadline timeouts by changing peer (tari-project#4649) fix(ci): libtor build on Ubuntu (tari-project#4644) chore: fix log (tari-project#4634) v0.38.2 fix(comms/rpc): detect early close in all cases (tari-project#4647) ...
…t#4690) Description --- - add validator registration sidechain feature - add basic VN signature - add output feature for VN reg - allow vn reg output feature in igor Motivation and Context --- DAN vn reg How Has This Been Tested? --- Existing tests
…tari-project#4692) Description --- - calculates validator node MR in `calculate_merkle_roots` function - allows tari_core and wallet to compile without base node feature - removes validator_node_mr param from `BlockHeader::from_previous` - removes validator_node_mr from NewBlocktemplate - removes validator node mr validation task from async validator - adds validator node mr validator to `check_merkle_roots` - removes unused get_validator_mr function from blockchain db - checks validator node mr in genesis block sanity check - removes panic from grpc conversion code Motivation and Context --- Validator node MR is created and checked in a different way from other merkle roots, this PR brings that code inline with other the current merkle root code + number of minor improvements. How Has This Been Tested? --- Existing tests - TODO: write validator node registration tests for blockchain db and block validators
…-project#4719) Description --- Adding the `getActiveValidatorNodes` method to `base_node_grpc_client` JS client. Motivation and Context --- For the `tari-explorer` web app (and other future usages, like cucumber integration tests), we want to retrieve a list of the active validator nodes from the base node JS client. The method is already implemented in the base node itself, this PR is only to allow calling it from the client. How Has This Been Tested? --- Manually from a modified version of the `tari-explorer` web app
Description --- * Created a new `lmdb` database for template registrations, with methods for inserting and fetching them. * Base layer stores in the database all templates that appear in new blocks. * New gRPC method `GetTemplateRegistrations` to retrieve all new templates since a specific block height Motivation and Context --- In [previous work](tari-project#4470) we added template registration to UTXO sidechain features. The next step is for the base layer to store and index all template registrations that appear in blocks, as well as provide a gRPC method to retrieve them. As the main query for templates will come from the base layer scanner in the Validator Node, the gRPC query method should allow to filter all new templates since a specific block height. How Has This Been Tested? --- Manually ran the base layer and perform a gRPC query via Postman.
get_shard_key was not using the correct database key when searching
* development: v0.38.5 feat: different default grpc ports for different networks (tari-project#4755) fix(core): broken doctests (tari-project#4763) ci: fix coverage job ci: run coverage on prs (tari-project#4738) fix(comms): fixes edge case where online status event does not get published (tari-project#4756) fix(dht/encryption): greatly reduce heap allocations for encrypted messaging (tari-project#4753) docs: explain the emission curve parameters (tari-project#4750) fix(comms/peer_manager): add migration to remove onionv2 addresses (tari-project#4748) fix(ci): add cargo cache, reduce Ubuntu dependencies and action on pull_request (tari-project#4757) feat(tariscript): adds ToRistrettoPoint op-code (tari-project#4749) fix: cli wallet cucumber (tari-project#4739) fix(clients): fix tari nodejs client proto paths (tari-project#4743) chore: disallow onion v2 (tari-project#4745) feat: change priority in mempool to take into account age (tari-project#4737) feat: trigger mempool sync on lag (tari-project#4730) fix(core): use compact inputs for block propagation (tari-project#4714) ci: deny dbg macro (tari-project#4740)
…evelopment chore: merge development into feature dan
…_reg response (tari-project#4788) Description --- Adds tx_id and template_address to create_template_registration method Motivation and Context --- Mainly helpful for displaying the template address in the vn cli. How Has This Been Tested? ---
Description --- Fix the VNs mmr. The output vector was in random order that resulted in the mmr being kind of random as well. So the miner and base node had different order of VNs and the blocks were invalidated. More VNs = less probability of passing. How Has This Been Tested? --- I manually registered 27 VNs and mined 100 blocks.
…quest (tari-project#4789) Description --- - adds UTXO and block info to get_template_registrations request - adds end_height to get_template_registrations request - Changes validator node registration key to <block_height, output_hash> - uses CompositeKey for all composite keys - Removed some unused (previous contract code) response types Motivation and Context --- DAN template scanner requires block hash info to determine last scanned height. Indexing template validations by block height enable more efficient retrievals NOTE: blockchain db will need to be resynced How Has This Been Tested? --- Manually - DAN block scanner
…ponent (tari-project#4818) **Still a work in progress** Description --- * Splitted `tari_base_node` and `tari_console_wallet` into `lib.rs` and `main.rs` * TBD: split `tari_miner` Motivation and Context --- To enable the new cucumber-rs integration tests on [tari-dan#103](tari-project/tari-dan#103), specifically to spawn base nodes and wallets, it's desirable to be able to treat them as native Rust functions exported in a `lib.rs` module. How Has This Been Tested? ---
* fix: batch rewind operations (tari-project#4752) Description --- Split rewind DbTx into smaller pieces. How Has This Been Tested? --- I did rewind on 20000+ (empty) blocks. * fix: fix config.toml bug (tari-project#4780) Description --- The base node errored when reading the `block_sync_trigger = 5` setting ``` ExitError { exit_code: ConfigError, details: Some("Invalid value for `base_node`: unknown field `block_sync_trigger`, expected one of `override_from`, `unconfirmed_pool`, `reorg_pool`, `service`") } ``` Motivation and Context --- Reading default config settings should not cause an error How Has This Been Tested? --- System level testing * fix(p2p/liveness): remove fallible unwrap (tari-project#4784) Description --- Removed stray unwrap in liveness service Motivation and Context --- Caused a base node to panic in stress test conditions. ``` thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: DhtOutboundError(RequesterReplyChannelClosed)', base_layer\p2p\src\services\liveness\service.rs:164:71 ``` How Has This Been Tested? --- Tests pass * fix(tari-script): use tari script encoding for execution stack serde de/serialization (tari-project#4791) Description --- - Uses tari script encoding (equivalent to consensus encoding) for `ExecutionStack` serde impl - Rename as_bytes to to_bytes as per rust convention. - adds migration to fix execution stack encoding in db Motivation and Context --- Resolves tari-project#4790 How Has This Been Tested? --- Added test to alert if breaking changes occur with serde serialization for execution stack. Manual testing in progress * feat: optimize transaction service queries (tari-project#4775) Description --- Transaction service sql db queries must handle `DieselError(DatabaseError(__Unknown, "database is locked"))`. This PR attempts to remove situations where that error may occur under highly busy async cirumstances, specifically: - Combine find and update/write type queries into one. - Add sql transactions around complex tasks. _**Note:** Partial resolution for tari-project#4731._ Motivation and Context --- See above. How Has This Been Tested? --- - Passed unit tests. - Passed cucumber tests. - ~~**TODO:**~~ System level tests under stress conditions. * feat: move nonce to first in sha hash (tari-project#4778) Description --- This moves the nonce to the front of the hashing order when hashing for the sha3 difficulty. This is done so that mining cannot cache part most the header and only load the nonce in. This forces the miner to hash the complete header each time the nonce chances. Motivation and Context --- Fixes: tari-project#4767 How Has This Been Tested? --- Unit tests all pass. * fix(dht): remove some invalid saf failure cases (tari-project#4787) Description --- - Ignores nanos for `stored_at` field in StoredMessages - Uses direct u32 <-> i32 conversion - Improve error message if attempting to store an expired message - Discard expired messages immediately - Debug log when remote client closes the connection in RPC server Motivation and Context --- - Nano conversion will fail when >= 2_000_000_000, nanos are not important to preserve so we ignore them (set to zero) - u32 to/from i32 conversion does not lose any data as both are 32-bit, only used as i32 in the database - 'The message was not valid for store and forward' occurs if the message has expired, this PR uses a more descriptive error message for this specific case. - Expired messages should be discarded immediately - Early close "errors" on the rpc server simply indicate that the client went away, which is expected and not something that the server controls, and so is logged at debug level How Has This Been Tested? --- Manually, * v0.38.6 * fix(core): only resize db if migration is required (tari-project#4792) Description --- Adds conditional to only increase database size if migration is required Motivation and Context --- A new database (cucumber, functional tests) has no inputs and so migration is not required. Ref tari-project#4791 How Has This Been Tested? --- * fix(miner): clippy error (tari-project#4793) Description --- Removes unused function in miner Motivation and Context --- Clippy How Has This Been Tested? --- No clippy error * test: remove cucumber tests, simplify others (tari-project#4794) Description --- * remove auto update tests from cucumber * rename some tests to be prefixed with `test_` * simplified two cucumber tests by removing steps Motivation and Context --- The auto update tests have an external dependency, which makes it hard to test reliably. They were marked as broken, so I rather removed them. There were two steps in the `list_height` and `list_headers` tests that created base nodes. Upon inspection of the logs, these base nodes never synced to the height of 5 and were not checked in the test, so were pretty useless and just slowed the test down How Has This Been Tested? --- npm test * v0.38.7 * feat: add deepsource config * fix(core): periodically commit large transaction in prune_to_height (tari-project#4805) * fix(comms/rpc): measures client-side latency to first message received (tari-project#4817) * fix(core): increase sync timeouts (tari-project#4800) Co-authored-by: Cayle Sharrock <CjS77@users.noreply.github.com> * feat: add multisig script that returns aggregate of signed public keys (tari-project#4742) Description --- Added an `m-of-n` multisig TariScript that returns the aggregate public key of the signatories if successful and fails otherwise. This is useful if the aggregate public key of the signatories is also the script public key, where signatories would work together to create an aggregate script signature using their individual script private keys. Motivation and Context --- To enhance the practicality of the `m-of-n` multisig TariScript. How Has This Been Tested? --- Unit tests Co-Authored-By: SW van Heerden swvheerden@gmail.com * feat(comms): adds periodic socket-level liveness checks (tari-project#4819) Description --- - adds socket-level liveness checks - adds configuration to enable liveness checks (currently enabled by default in base node, disabled in wallet) - update status line to display liveness status Motivation and Context --- Allows us to gain visibility on the base latency of the transport without including overhead of the noise socket and yamux How Has This Been Tested? --- Manually * fix(core): dont request full non-tip block if block is empty (tari-project#4802) Description --- - checks for edge-case which prevents an unnecessary full candidate block request when block is empty. Motivation and Context --- A full block request for empty block is not necessary as we already have all the information required to construct the candidate block. This check was missing from the branch where the candidate block is not the next tip block. How Has This Been Tested? --- Co-authored-by: Martin Stefcek <35243812+Cifko@users.noreply.github.com> Co-authored-by: Hansie Odendaal <39146854+hansieodendaal@users.noreply.github.com> Co-authored-by: SW van Heerden <swvheerden@gmail.com> Co-authored-by: stringhandler <mikethetike@tari.com> Co-authored-by: CjS77 <CjS77@users.noreply.github.com>
…project#4845) * add remaining logic to build on ConsensusConstants grpc interface * address some PR comments * correct grammar error * cargo clippy * further refactoring * add constructor to TransactionOutput type * cargo fmt
Description --- Removes unused config from validator node Motivation and Context --- Fixes tari-project#4814 How Has This Been Tested? --- Validator node starts up
…tants (tari-project#4856) Description --- In this PR, we refactor the method `get_constants` of `BaseNodeClient` to request a block height input. Motivation and Context --- In order to get `ConsensusConstants` values at the DAN layer, it is useful to be able to request the former using a block height, for synchronization purposes. This PR addresses this scenario. How Has This Been Tested? ---
* development: (52 commits) chore: better help for seed-words command (tari-project#4885) fix(ci): resolve windows binary builds (tari-project#4883) fix(ci): correct ARM64 builds (tari-project#4876) fix(comms/peer_manager): fix possible panic in offline calc (tari-project#4877) feat!: impl final tari pow algorithm (tari-project#4862) fix(ci): selectively revert resolver for arm64 builds (tari-project#4871) chore(deps): bump actions/checkout from 2 to 3 (tari-project#4873) fix: delete orphans if they exist (tari-project#4868) chore: replace manual implementation of char methods (tari-project#4864) chore: fix potentially buggy split of string into lines (tari-project#4863) fix(ci): update GHA set-output plus dependabot schedule for GHA (tari-project#4857) fix(base-node): use less harsh emoji for unreachable node (tari-project#4855) fix(core): add txo version checks to async validator (tari-project#4852) feat: add static lifetime to emission amounts calculation (tari-project#4851) v0.38.8 feat: add opcode versions (tari-project#4836) fix: remove clear_on_drop dependency (tari-project#4848) fix(base-node): use Network::from_str to parse network in cli (tari-project#4838) ci: remove circleci test: add cucumber critical (tari-project#4823) ...
…ocalnet (tari-project#4879) Description --- Changed the `validator_node_timeout` consensus constant from `0` to `100` in the `localnet` network. Motivation and Context --- We use the `localnet` network in the `tari-dan` project `cucumber-rs` integration tests. The current value for the `validator_node_timeout` constant in that network is `0`, which makes the validator node registrations to never be valid. It should be a reasonable value, similar to the `igor` network. How Has This Been Tested? --- With this change, in the `tari-dan` project `cucumber-rs` integration tests does now list registered validator nodes
…oject#4880) Description --- - Remove unused `get_committee` function from base layer - add test to KeyPrefixCursor to confirm it works as expected Motivation and Context --- `get_committee` would panic because it made simultaneous accesses to the same transaction by creating multiple KeyPrefixCursors. Since it is unused and implemented on the DAN layer, the decision was taken to remove it. How Has This Been Tested? --- Added new test to KeyPrefixCursor BREAKING CHANGE: grpc interface no longer has get_committee call
chore: merge development branch into feature-dan
* feature-dan: (21 commits) fix(core)!: remove unused get_committees call from base node (tari-project#4880) fix: correct value for validator_node_timeout consensus constant in localnet (tari-project#4879) feat: add block height to input request to get network consensus constants (tari-project#4856) fix: remove unused config for validator node (tari-project#4849) feat: add missing fields to grpc consensus constants interface (tari-project#4845) chore: merge development into feature-dan (tari-project#4815) refactor: split tari_base_node and tari_console_wallet into a lib component (tari-project#4818) fix(core)!: adds utxo and block info to get_template_registrations request (tari-project#4789) fix: computation of vn mmr (tari-project#4772) fix(wallet/grpc): add transaction id and template_address to template_reg response (tari-project#4788) fix: fix get shard key (tari-project#4744) feat(core): store and fetch templates from lmdb (tari-project#4726) fix: fix validator node registration logic (tari-project#4718) feat(base_node_grpc_client): add getActiveValidatorNodes method (tari-project#4719) fix after merge fix(core): bring validator node MR inline with other merkle root code (tari-project#4692) feat(core): add validator registration sidechain feature (tari-project#4690) fix merge issues feat: add grpc to get shard key for public key (tari-project#4654) feat: add validator node registration (tari-project#4507) ...
sdbondi
changed the title
sdbondi
changed the title
stringhandler
approved these changes
Nov 14, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Merged
feature-danbranch intodevelopment