Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add epoch blocks to disable old type blocks #955

Merged
merged 11 commits into from
Jul 13, 2018
Merged

Conversation

PlasmaPower
Copy link
Contributor

This PR adds epoch blocks, which are a variant of state blocks. They will be used to prevent future old type (non-state) blocks. This code may also be used for future upgrades.

How to detect if a state block is a epoch block:

  • Its link field is epoch_link (currently "epoch v1 block" then a bunch of null bytes)
  • Its balance is the same as the previous block for the account (or 0 if previous is 0)

Additional requirements of an epoch block (for more details see rai::ledger_processor::epoch_block_impl):

  • Its account must not be the burn account
  • It is signed by epoch_signer (currently the genesis account) instead of its account
  • Its representative is the same as the previous block for the account (or the burn account if previous is 0)
  • Its account's version must be 0

An epoch block will change the account version from 0 to 1, which means:

  • It cannot be followed by old type blocks (like how state blocks already work)
  • Any future sends will have a min_version of 1
    • These cannot be received by old type blocks
    • When received by a state block, this automatically upgrades the account to version 1

We'll make a script to generate epoch blocks for any currently open accounts and any unopened account with pending sends (except the burn account).

@@ -110,13 +110,14 @@ std::unique_ptr<rai::block> deserialize_block (MDB_val const &);
/**
* Latest information about an account
*/
#pragma pack(push, 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may not be a problem in this case, but this does mean we need to worry about unaligned access as we maintain these classes (i.e. no atomics in the future in these structs, no assignment from ptr-to-member, and for ARM, slow code for unaligned access may be generated).

An alternative is maybe to pack manually instead of std::copy'ing the struct pointer.

* Mark epoch blocks in history

* Extra info for raw

* Remove epoch block from non-raw RPC history

* Move raw check
Copy link
Contributor

@SergiySW SergiySW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if epoch signer can be MuSig of top reps for later epochs

@PlasmaPower
Copy link
Contributor Author

I think we can just use active graph for future epochs (so we wouldn't need epoch blocks at all).

Copy link
Contributor

@rkeene rkeene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only issues I had with these were cosmetic or preference, and those can all be worked out in a different changeset later -- merge it to master and get to testing.

@PlasmaPower PlasmaPower merged commit 2608433 into master Jul 13, 2018
@gr0vity
Copy link

gr0vity commented Jul 14, 2018

With the merging of this PR, the genesis account now is able to sign blocks on my chain.
In my eyes this is a major change that needs a lot of thought.

Prior to this PullRequest, my private key is the only key that can modify my own chain.
This is a sound concept. I am the only owner of my chain and the only one who can change my chain.
With epoch blocks this changes!

After this PullRequest gets merged, the genesis account now is able to sign (epoch) blocks on my behalf.
Doesn't this go against decentralisation ? Can't this be exploitet as an additional attack vector ? What if the genesis private key gets lost or stolen ?
Who tells me that in the future epoch blocks will not be able to modify my representative, my balance, ... ?

Please explain a little more, why this change is not dangerous now and in the future.
Thanks

@clemahieu
Copy link
Contributor

clemahieu commented Jul 14, 2018

As a starting point, in the description it says the exactly one condition this block can be inserted. Most specifically it cannot involve a balance transfer in any form; the only account state it changes is the account’s version.

The reason we’re looking at these markers is as a way to upgrade the ledger, since all accounts are asynchronous we need an asynchronous form of chain upgrades. We’re not able to say “upgrade at block X” as in bitcoin

If you have a way to mark cold-wallet or lost-key accounts I’m open to suggestions. There isn’t a way to determine the difference between these so this seemed like a good solution.

@gr0vity
Copy link

gr0vity commented Jul 14, 2018

I am okay with upgrading the ledger.
I am okay with accepting the epoch block myself.

What I dislike if the fact that a different account (genesis account) can alter my chain.
Right now the conditions for epoch blocks to be signed are balance and rep must be the same .
But you made these rules.. and you could change these rules.
Who prevents these rules from changing? Now that the genesis can sign blocks on my behalf... If some day these rules change, well I see a problem. I don't say they will. But they could... Or can't they change ?

So can we say that epoch blocks can never ever be used to alter my account balance and representative ?
And can we say that there is no bug in the implementation that could lead to a balance or rep change ?
If so, it's probably the best solution. Just saying.
Maybe it's only me, but I see this is a major change of concept which needs careful thought and review.

Do the majority of reps need to use the latest version to support epoch blocks ? Could I refuse to vote an epoch block as valid when i remain on v14.2 ? If the majority of reps need to upgrade to the latest version for epoch blocks to be valid, I guess it's okay the way you do it.

Maybe you could use reps to vote on a 1-time account with rights to sign only v1 epoch blocks instead of giving gensis full access to sign every epoch block.

I think what's important is, that the majority of reps need to move to the new version for epoch blocks to be voted as a valid block.

@PlasmaPower
Copy link
Contributor Author

If the majority of reps don't move, the new chain won't work, because nothing will reach quorum.

FCherepanov added a commit to artem-kruglov/raiblocks that referenced this pull request Jul 23, 2018
* Spring cleanup of root directory (nanocurrency#784)

* Lies.

* Only push nano-ci docker image on master

Also disables attempting to cache the nano images, because --cache-from
can't work with two stage builds, and the default cache will speed up
the second beta build.

* Remove disabled debug command (nanocurrency#788)

* Automatically vacuum database if it gets too large in the Docker container (nanocurrency#792)

* Remove some unused captures and locals, and constructor initialization list reordering (nanocurrency#786)

* Work generation for state blocks in process RPC (nanocurrency#785)

* Optionally disable local work when work peers are down (nanocurrency#791)

* Log ledger rollbacks votes

* Bool for faster new_winner calculation

* typo

uint vs unsigned int for msvc

* Log each block votes weight

* Using a single transaction to check all wallets and checking for account existence in the same transaction.

* Adding optional confirm_req answers

If tthis is request for forked block/account representative knows about

* Using ledger.successor to retrieve correct block

* Remove asserts in successor function

For debug mode: This is fine to not have successor

* Remove not required condition for confirm_req

for confirm_req options

* Add a query to see if votes are uncontested.

* Protocol version constants (nanocurrency#805)

* Queueing rollbacks through the block processor instead of in an IO thread.  process_receive_many can block on the database lock and might cause IO thread starvation.

* Formatting.

* Cleaning up and simplifying fork resolution process.

* Holding reference to node by weak_ptr so it doesn't suspend the shutdown process.

* Pushing blocks to the front of the queue because:
1) During bootstrap it will be received frontier-backwards to newer blocks are less likely to land in unchecked
2) Rollbacks will be resolved quicker

* Setting minimum protocol version to network version 7 and depeering any node lower than that.

* logging updates (nanocurrency#807)

* logging updated

Moved logging for unknown block type under network_packet loggin

* logging updated

various logging messages have been moved into the appropriate config switches

* Formatting

* Toning down connection spam in log.

* Fix OSX build on Travis

* Streamline bootstrap logging to remove non-useful information.

* Adding logging to block processing queue so we can see it's doing something.

* Removing dead code.

* Adding priority queue so blocks voted to be rolled back get removed quicker.

* Removing block_processor_item type and just using a block shared pointer since forced items are in a separate queue now.

* Bumping version numbers.

* Windows Specific Logging (nanocurrency#811)

Silence Windows Specific Common Errors while bootstrapping

* Make elections when bootstrapping more resilient (nanocurrency#810)

This does a two things for elections while bootstrapping:

- Space out the initial confirm_reqs to reduce packet loss
- Retry confirm_reqs for big reps if we don't get a response

This also makes rep_crawler more aggressive, so that we have more reps to confirm_req.

* Add work and signature to account_history response if raw is true

* Fix potential segfault in announce_votes

* Disallow account variations passing checksum test (nanocurrency#797)

* Fix state block send callback amount to not be inverted (nanocurrency#796)

* Logging improvements (nanocurrency#779)

* Add balance option for blocks_info RPC action (nanocurrency#794)

* Minor RPC block_create changes (nanocurrency#804)

* Fetching account balance & previous for send blocks

if aren't given directly

* Account field is not required for state blocks

as it has private/public key

* If there is no link in request use source or destination

* Double check current balance if previous block is specified

* Link can be 0 for changing representative

* Simplifying filter loop, correctly handling removal of last item, and not skipping check on item swapped in from the back.

* Don't call broadcast_reps with empty list.

* Bumping developmen/master version number.

* Fixing RPC block_create test (nanocurrency#814)

"previous" field can be auto-filled now

* Use default values on success/error to give accurate errors instead of ptree exceptions when tests fail (nanocurrency#815)

* Correct mdb table comments with  Doxygen conversion (nanocurrency#816)

* Encapsulating bootstrap_initiator internal data.

* Don't hold bootstrap attempt open while forks are being resolved.

* Using a low-overhead timeout mechanism using only an atomic int instead of deadline_timer which can involve a syscall/read.

* Allow zero balance in block_create for state block sends (nanocurrency#817)

* Fixed 'account' field for state send subtype in 'account_history' RPC call (nanocurrency#819)

* Fix formatting.

* QT 5.9.5 @rpath is set correctly so we don't need to install_name_tool every framework.

* Remove unused function and captures (nanocurrency#822)

* ledger::supply is a hotspot, so we memoize and update periodically. (nanocurrency#824)

* Adding manual block confirmation function.

* Adding block_confirm RPC.

* Moving block callback post-confirmation.

* Fixing formatting.

* Merge branch 'sentinal_mode-2' into sentinel-squash

* Eliminating duplicate quorum check code.

* Checking if resolved at call site instead of within callback.

* Statistics support (nanocurrency#759)

* Logging quorom weight received.

* Fix handling of invalid wallet work (nanocurrency#832)

* Convert dynamic casts to static casts (nanocurrency#831)

* Add RPC block_hash action (nanocurrency#830)

* Add logging to help track down a possible race (nanocurrency#837)

* Resolve breaking changes in Boost 1.67 (nanocurrency#829)

* Simplify rollback visitor for send blocks (nanocurrency#838)

* Expanding the window in which blocks are rebroadcast to the network.
Changing from 60 to 300 second retention and also tracking the last 8000 blocks.

* Removing private addresses from live network peers

https://tools.ietf.org/html/rfc1918
https://tools.ietf.org/html/rfc6598
https://tools.ietf.org/html/rfc4193

* Rewrite unsynced (fixes bulk_push_client)

* Add logging

* Temporarily disable bulk_pull_logging check for bulk push logging

* Revert "Temporarily disable bulk_pull_logging check for bulk push logging"

This reverts commit 26b1823.

* Fixing merge artifact.

* Prevent log deletion by inactive_node (nanocurrency#802)

* Prevent log deletion by inactive_node

if logging.max_size > default value

* Clang formatting

* Fix typo keeping bootstraps open during exit (nanocurrency#842)

* Push missing blocks back to the peer we requested frontiers from since this is the node missing the blocks. (nanocurrency#844)

* Fix inverted check in scan_receivable causing spurious asserts (nanocurrency#846)

* Rep votes don't need to be computed during election construction since they're always computed before broadcasting the winner. (nanocurrency#849)

Removing unused transaction parameters in election and active_transactions.

* Extracting root_exists function to check if the root block or account exists.

* Supporting nano_ and xrb_ prefixes. (nanocurrency#854)

* Rebroadcast blocks via process RPC (nanocurrency#852)

* The process RPC was reimplementing block processing but didn't properly rebroadcast blocks it processed.  This converts the RPC to just pass the block off to the block processor.  The processing result can be retrieved via logging.

* Fixing formatting.

* Supporting nano_ and xrb_ prefixes. (nanocurrency#854)
 (reverted from commit ec7432b)

* Supporting nano_ and xrb_ prefixes.
Deprecating support for v1 addresses.
 (nanocurrency#854)
 (reverted from commit ec7432b) (reverted from commit ae23b77)

* Renaming list_sqrt to list_fanout so the naming isn't tied to one form of rebroadcast fanout sizing.
Changing data structure to a deque since we don't need the entries to be contiguous.

* Halving network fanout since the previous value was extremely conservative.

* Simplifying send_keepalive test which won't compile on some STL implementations due to an issue with deque iterators.

* Fixed formatting.

* Don't put key in wallet otherwise it's a race condition between block_create and the wallet internally receiving the block.

* Using a less-than operator instead of greater-than and using vote_info instead of a tuple.

* Moving vote_replay code in to vote_processor as a more appropriate place to hang it.
Only returning vote code instead of vote used since we don't need to track the vote used and might not always have access to it.
Vote replay code now works appropriately if there's an active election for the block.

* Continuing to announce votes until quorum is met or the block can no longer be inserted.

* Adding logging if blocks aren't being confirmed for an excessive amount of time.

* Fixing formatting.

* Bumping protocol version on master since we have a V13 stabilization branch.

* Fix stateblock receive account field to be consistent with legacy blocks (nanocurrency#820)

* Fix stateblock receive account field to be consistent with legacy blocks

* Add account field to output of account_history RPC call

* Only look up account if it is not provided in account_history RPC call

* Statistics pane under Advanced in Qt wallet (nanocurrency#864)

* optional use_peers param to generate_work rpc (nanocurrency#862)

* Queuing a wallet action in a background thread is unnecessary. (nanocurrency#876)

* Simplify and improve performance of search_pending. (nanocurrency#866)

* Simplify and improve performance of search_pending.
Invert loop iteration since the number of accounts in the wallet is likely smaller than the number of pending entries.
Automatic block confirmation already checks if the destination is in a local wallet so we don't need to re-implement this.
Re-searching the pending table for source blocks for the confirmed account can be better implemented in the future with an account dirty flag or queued block insertion.
Since we don't need a callback after confirmation we don't need the search code in its own class.

* Using block_confirm instead of doing it manually.

* Fix Appveyor (nanocurrency#875)

* Changing frontier_count to account_count (nanocurrency#879)

* Count the accounts table instead of frontiers table to fix frontier_count RPC

* Renaming frontier_count to account_count.
Retaining old frontier_count RPC for compatibility.

* Making callback outside of work_pool queue lock.

* Fix wallet send action so it respects generate_work parameter and also fix test so to pass false to generate work.

* Run work cancel tests in a loop since it's a race condition to successful cancellation.

* Using queue_wallet_action for ensure working is generated for newly inserted keys.

* Use a common scheme for function naming and name some as blocking to indicate they block while work is being generated.

* Removing work_fetch as it's only used in one place.

* Fixing formatting.

* Update rpc.cpp

generate_work was renamed, one instance was missed

* Add DNS resolution support for work peers (nanocurrency#865)

* Fix typo: quorom -> quorum (nanocurrency#881)

* Fix type, quorom -> quorum

* Upgrade config JSON

* Fix syntax error

* Fix formatting

* Don't insert a block in to the processing queue if it's already marked as active which implies it's already been added to the queue.

* Update node.cpp

Fixed nanocurrency#861/nanocurrency#881 as the config version was incremented but not saved to config properly

* Only send one confirm_req instead of both.  Since v11 nodes have responded with the block they have for the root associated with the block so both aren't needed.

* Rename Doxygen name to Nano (nanocurrency#889)

* Moving vote deserialization on to the vote class.

* Initialize state member in block_counts constructor

Fixes nanocurrency#900

* Update ledger stats (nanocurrency#902)

* Remove unused transaction from confirmation_history (nanocurrency#903)

* Option to force fork resolution in RPC process (nanocurrency#892)

* Option to force fork resolution in RPC process

* Enabled control should be required

for manual fork resolution

* Deleting forced block from elections

* Add stats for outgoing bulk push (nanocurrency#905)

* Making verification a member of the vote class.

* Hoisting calls to balance visitor up to reduce recursion.

* Update common.cpp

clang formatting fix

* Calculating tally only once.

* Changing local block and vote before confirmation is triggered.

* Removing unused arguments.

* Fixing formatting.

* Resolving forks as part of the node instead of part of the bootstrap process.

* Fix formatting.

* Disable compute_rep_votes in config

if enable_voting set to false, wallet shouldn't compute its account votes

* Fork test sometimes needs more iterations.

* Stateblock Canary Check Cleanup

Stateblocks are now being parsed and generated, Canary blocks no longer matter since only nodes that understand them are still on the network

* Removing more canary block code.

* Remove unused process_result related code (nanocurrency#911)

* Throttling pull requests if blocks can't be processed quick enough which was causing large amounts of memory usage in the block_processor.

* Removing excess logging.

* Bootstrap connection cleanup (nanocurrency#912)

* Deserialize in to receive buffer without any pointer arithmetic.

* Creating rai socket class that has timeout capability encapsulated.

* Putting message header in its own class and removing buffer pointer arithmetic when parsing bootstrap messages.

* Converting bootstrap_server connection to a rai::socket to ensure timeouts are used when sending data.

* Version bump

Bumping protocol version on master since we have a V14 stabilization branch

* Trim whitespaces in certain input fields in Qt wallet to avoid confusion on paste

* Adding public keys for release signing

* Voting winner patch (by Lee Bousfield) (nanocurrency#916)

* Voting winner patch (by Lee Bousfield)

* Missed from patch

* Revert "Voting winner patch (by Lee Bousfield) (nanocurrency#916)"

This reverts commit e1b9c1f.

* shared_ptr <rai::block> comparison as hash

* CLI unchecked_clear (nanocurrency#915)

and optional unchecked_clear for "vacuum" & "snapshot"

* Update utility.cpp (nanocurrency#924)

support lower end devices that can't handle the existing code.

* Simplify search_pending (nanocurrency#922)

Voting for each pending block separately

* Correct account fields in QT wallet (nanocurrency#914)

* Assign nodes a persistent ID

* bootstrap improvements

Stale forks should be able to be evicted through bootstrapping, even if they've recently come off the network.

* Update node.cpp

clean up verbose logging to std::cerr by default

* Fixing upgrade path for node_id logging

in the config

* Don't count state blocks twice in stats (nanocurrency#927)

* Accurate rep count in logs when sending confirm req (nanocurrency#926)

* RPC wallet_change_seed functionality should match CLI (nanocurrency#925)

Backports nanocurrency#637.

* Set permissions on backup files so only owner can read and update the backup (nanocurrency#929)

* GTEST for MSVC 15.5 compatibility

* Restart of Beta network (nanocurrency#936)

* New beta network genesis

* Adding beta network preconfigured reps

* Update beta.json

Correct RPC port for default beta config.json

* Replace call operator with notify function (nanocurrency#940)

* Add missing version_put (nanocurrency#939)

fixes nanocurrency#919

* Update live.json & beta.json (nanocurrency#938)

for Docker images

* State blocks output for QT wallet "Create Block" (nanocurrency#934)

* State blocks output for QT wallet "Create Block"

* Clang formatting

* Move boost_log_setup before boost_regex when linking (nanocurrency#933)

Without this when compiling rai_node on fedora 28 I get many undefined references starting with:
```
[ 68%] Linking CXX executable rai_node
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libboost_log_setup.a(init_from_settings.o): Funktiossa ”boost::log::v2s_mt_posix::(anonymous namespace)::default_syslog_sink_factory<char>::create_sink(boost::log::v2s_mt_posix::basic_settings_section<char> const&)”:
(.text+0x1b72): määrittelemätön viite kohteeseen ”boost::log::v2s_mt_posix::sinks::syslog_backend::set_severity_mapper(boost::log::v2s_mt_posix::aux::light_function<boost::log::v2s_mt_posix::sinks::syslog::level (boost::log::v2s_mt_posix::record_view const&)> const&)”
```

* Confirm req to all peers if reps list is empty (nanocurrency#937)

or online stake is less than required quorum

* Update README.md (nanocurrency#941)

Road-map and Twitter Link updates

* std::error_code and std::expected implementation

* Remove store_entry class since it's effectively just a pair of rai::mdb_val

* Speed up travis builds: enable ccache, cache cargo and reduce test timeouts (nanocurrency#957)

* Do not hang trying to upload to docker in unofficial repos

docker push hung asking for a password if the password environment
variable was not set. With this change it fails immediately.

* Enable ccache on linux and osx

* Cache homebrew downloads

This does not avoid the compilation of the homebrew packages but stores
the downloaded sources, speeding up the installation process.

* Enable cargo caching

* Cache load-tester cargo binaries

* Reduce default timeout per test suite to 120 seconds

One of the tests hangs in the CI system, reducing the timeout to have
quicker feedback. It can be re-adjusted in the future if 120 is not
enough for a successful run.

* Fix logging config serialization

* Raw history changes (nanocurrency#943)

* Return "balance" in raw history

for state blocks

* Return "previous" in raw history

* Remove previous for open block

* Added "bootstrap_connections_max" to beta.json (nanocurrency#960)

* CLI refactoring

* Fix the work value of the beta network genesis block (nanocurrency#963)

* Fix deterministic wallet keys in boost 1.67 or above (nanocurrency#967)

* Fix deterministic wallet keys in boost 1.67 or above

* Match existing casting idiom

* Fix several core_test issues (nanocurrency#959)

* Fix test rpc.version

* Sometimes election can start before block_processor commit

* Attempt to process confirmed block if it's not in ledger yet

* Fix QT block creation tests

* Enable tests upgrade_v1_v2 & send_single_many_peers

* Increase iterations for fork_pre_confirm

* Fix -Wconversion-null warnings

* Disable confirm_if_quorum for already confirmed election

* Fix error with 0 key in wallet store

* Redo "Disable confirm_if_quorum"

* Increase possible synchronizing time for QT wallet

* Log ID for each RPC call (nanocurrency#951)

Now it's logged only on completion

* Fix more core tests

* Fix rpc.wallet_ledger

* Fix more time comparison in rpc tests

* Disable node.bootstrap_connection_scaling

* Fix message.confirm_ack_serialization

* Initialize booleans when not explicitly set (nanocurrency#971)

* Add epoch blocks to disable old type blocks (nanocurrency#955)

* Add version field to account_info and min_version to pending_info

* Add epoch block support to ledger

* Add epoch block settings to config, and remove state canaries

* Add test for epoch blocks

* Add receive upgrade test for epoch blocks

* Ignore old nodes on the beta network

* Add min_version to RPC pending and wallet_pending

* Add account_version field to account_info RPC

* Fix formatting

* Remove epoch block config (de)serializing

* Mark epoch blocks in history (nanocurrency#968)

* Mark epoch blocks in history

* Extra info for raw

* Remove epoch block from non-raw RPC history

* Move raw check

* Fix upgrade_v11_to_v12 when already upgraded

* Fix rpc.version test

* Fix system.generate_send_existing

* Require core tests to pass in Travis (nanocurrency#969)

* Fix rai::block_store::upgrade_v11_to_v12

* modify type error: age => count (nanocurrency#932)

* Hopefully fix rpc.work_get

* Move rai/node library definition to separate CMakeLists.txt

* Move rai/lib library definition to separate CMakeLists.txt

* Add latest version FindBoost.cmake

* Move rai_node to a separate CMakeLists.txt

* Move secure into its own folder

* Move ed25519 to its own CMakeLists.txt

* Avoid compiling rai_lib twice

This was apparently done to simplify the linking of the tests but it is
not needed if the library is compiled as STATIC as it seems to be
currently. This is because CMAKE_SHARED_LIBS is set to off by default.

* Fix compilation of core_test

* Fix miniupnpc include directories from the main CMakeLists.txt

* Fix compilation of GUI

 * Move badly placed icon from secure to rai_wallet
 * Fix includes
 * Fix dependencies

* Add missing dependencies to lmdb

* Fix compilation of GUI and tests

* Another pass removing left-overs in CMakeLists.txt

* Set exe linker flags globally

* Fix linux compilation: link against threading and dl

* Move all GUI specific code to the same if

* Only link lmbd dependencies to lmdb

* Move ACTIVE_NETWORK definition after option declaration

It was being used before its declaration

* Restore compiler extensions (fixes compilation on windows)

* Use custom lmdb

* Fix dependency of wallet on WinExtras and include

* Fix compilation and packaging in appveyor

* Build install target to build everything necessary for packaging
* Download vc_redist before building because the install target requires
  it

* Guard compilation of qt_test under RAIBLOCKS_TEST

* Start commenting out extra link libraries, include directories, etc.

* Group GUI declarations

* Add missing dependency secure -> boost

* Start modernizing cmake code

* Fix compilation of node

* Group all gtest code under if (RAIBLOCKS_TEST)

* Remove left-overs in CMakeLists.txt

* Comment out platform libs, to be added later if needed

minor cleanup of platform_wallet_libs

* Indent

* Define RAIBLOCKS_VERSION MAJOR and MINOR only where needed

* Force cmake to store libraries and executables in the root of the build dir

This fixes the tests because they could not find the rai_node executable

* Unify file extension list between check-commit-format and clang-format-all

* Apply clang-format-all.sh

* Change lmdb reference to official repo

And point to the correct branch

* Fix wallet_work_get

* Hopefully fix wallet.seed_work_generation

* Fix possible wallet locked errors in RPC (nanocurrency#977)

send, receive, change

* Fix peer_container.split

* Remove unnecessary observer subscriptions to reduce a bit of locking overhead
@clemahieu
Copy link
Contributor

At the beginning of the year we already demonstrated we won't alter account balances or representatives, this wasn't just words it was actions.

Reps already need to be upgraded because, as I already said, epoch blocks need to be explicitly programmed in to the node beforehand otherwise an old node will reject them as invalid.

Every code commit for every piece of software in the world is a possible rule change, that's how code works. That's also why this project is open source so everyone can verify what's being changed and can make sure it's in line with the spirit of the project.

This upgrade needs to be done and I'm not interested in dragging it out to pontificate on future scenarios that aren't going to happen. Projects that can't evolve will die and we're going to evolve the Nano protocol to survive and serve its purpose as a store and transfer of value.

If you see a problem with this actual code in this commit let me know.

bbedward added a commit to BananoCoin/banano that referenced this pull request Aug 24, 2018
* Move boost_log_setup before boost_regex when linking (nanocurrency#933)

Without this when compiling rai_node on fedora 28 I get many undefined references starting with:
```
[ 68%] Linking CXX executable rai_node
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libboost_log_setup.a(init_from_settings.o): Funktiossa ”boost::log::v2s_mt_posix::(anonymous namespace)::default_syslog_sink_factory<char>::create_sink(boost::log::v2s_mt_posix::basic_settings_section<char> const&)”:
(.text+0x1b72): määrittelemätön viite kohteeseen ”boost::log::v2s_mt_posix::sinks::syslog_backend::set_severity_mapper(boost::log::v2s_mt_posix::aux::light_function<boost::log::v2s_mt_posix::sinks::syslog::level (boost::log::v2s_mt_posix::record_view const&)> const&)”
```

* Confirm req to all peers if reps list is empty (nanocurrency#937)

or online stake is less than required quorum

* Update README.md (nanocurrency#941)

Road-map and Twitter Link updates

* std::error_code and std::expected implementation

* Remove store_entry class since it's effectively just a pair of rai::mdb_val

* Speed up travis builds: enable ccache, cache cargo and reduce test timeouts (nanocurrency#957)

* Do not hang trying to upload to docker in unofficial repos

docker push hung asking for a password if the password environment
variable was not set. With this change it fails immediately.

* Enable ccache on linux and osx

* Cache homebrew downloads

This does not avoid the compilation of the homebrew packages but stores
the downloaded sources, speeding up the installation process.

* Enable cargo caching

* Cache load-tester cargo binaries

* Reduce default timeout per test suite to 120 seconds

One of the tests hangs in the CI system, reducing the timeout to have
quicker feedback. It can be re-adjusted in the future if 120 is not
enough for a successful run.

* Fix logging config serialization

* Raw history changes (nanocurrency#943)

* Return "balance" in raw history

for state blocks

* Return "previous" in raw history

* Remove previous for open block

* Added "bootstrap_connections_max" to beta.json (nanocurrency#960)

* CLI refactoring

* Fix the work value of the beta network genesis block (nanocurrency#963)

* Fix deterministic wallet keys in boost 1.67 or above (nanocurrency#967)

* Fix deterministic wallet keys in boost 1.67 or above

* Match existing casting idiom

* Fix several core_test issues (nanocurrency#959)

* Fix test rpc.version

* Sometimes election can start before block_processor commit

* Attempt to process confirmed block if it's not in ledger yet

* Fix QT block creation tests

* Enable tests upgrade_v1_v2 & send_single_many_peers

* Increase iterations for fork_pre_confirm

* Fix -Wconversion-null warnings

* Disable confirm_if_quorum for already confirmed election

* Fix error with 0 key in wallet store

* Redo "Disable confirm_if_quorum"

* Increase possible synchronizing time for QT wallet

* Log ID for each RPC call (nanocurrency#951)

Now it's logged only on completion

* Fix more core tests

* Fix rpc.wallet_ledger

* Fix more time comparison in rpc tests

* Rewriting RPC with error codes

* Clang formatting

* Clang formatting

* Merge RPC chain & RPC successors

* Merge mrai_to_raw+krai_to_raw+rai_to_raw & mrai_from_raw+krai_from_raw+rai_from_raw

* Small changes

* Reduce code with wallet_impl

* Reduce code with account_impl

* Formatting

* Several errors

* Reduce code size with count_impl

* count_optional_impl

* Formatting

* Disable node.bootstrap_connection_scaling

* Fix message.confirm_ack_serialization

* Initialize booleans when not explicitly set (nanocurrency#971)

* Add epoch blocks to disable old type blocks (nanocurrency#955)

* Add version field to account_info and min_version to pending_info

* Add epoch block support to ledger

* Add epoch block settings to config, and remove state canaries

* Add test for epoch blocks

* Add receive upgrade test for epoch blocks

* Ignore old nodes on the beta network

* Add min_version to RPC pending and wallet_pending

* Add account_version field to account_info RPC

* Fix formatting

* Remove epoch block config (de)serializing

* Mark epoch blocks in history (nanocurrency#968)

* Mark epoch blocks in history

* Extra info for raw

* Remove epoch block from non-raw RPC history

* Move raw check

* Fix upgrade_v11_to_v12 when already upgraded

* Fix rpc.version test

* Fix system.generate_send_existing

* Require core tests to pass in Travis (nanocurrency#969)

* Formatting

* hash_impl

* Edit hash_impl to accept "block"

* rpc_control_impl

* amount_impl

* Doule write issue

* Typo

* Typo

* Fix rai::block_store::upgrade_v11_to_v12

* threshold_optional_impl

* Simplification

* Simplification

* Fix

* work_optional_impl

* Remove double check

* Simplification

* Return "account" in history

* modify type error: age => count (nanocurrency#932)

* Hopefully fix rpc.work_get

* Move rai/node library definition to separate CMakeLists.txt

* Move rai/lib library definition to separate CMakeLists.txt

* Add latest version FindBoost.cmake

* Move rai_node to a separate CMakeLists.txt

* Move secure into its own folder

* Move ed25519 to its own CMakeLists.txt

* Avoid compiling rai_lib twice

This was apparently done to simplify the linking of the tests but it is
not needed if the library is compiled as STATIC as it seems to be
currently. This is because CMAKE_SHARED_LIBS is set to off by default.

* Fix compilation of core_test

* Fix miniupnpc include directories from the main CMakeLists.txt

* Fix compilation of GUI

 * Move badly placed icon from secure to rai_wallet
 * Fix includes
 * Fix dependencies

* Add missing dependencies to lmdb

* Fix compilation of GUI and tests

* Another pass removing left-overs in CMakeLists.txt

* Set exe linker flags globally

* Fix linux compilation: link against threading and dl

* Move all GUI specific code to the same if

* Only link lmbd dependencies to lmdb

* Move ACTIVE_NETWORK definition after option declaration

It was being used before its declaration

* Restore compiler extensions (fixes compilation on windows)

* Use custom lmdb

* Fix dependency of wallet on WinExtras and include

* Fix compilation and packaging in appveyor

* Build install target to build everything necessary for packaging
* Download vc_redist before building because the install target requires
  it

* Guard compilation of qt_test under RAIBLOCKS_TEST

* Start commenting out extra link libraries, include directories, etc.

* Group GUI declarations

* Add missing dependency secure -> boost

* Start modernizing cmake code

* Fix compilation of node

* Group all gtest code under if (RAIBLOCKS_TEST)

* Remove left-overs in CMakeLists.txt

* Comment out platform libs, to be added later if needed

minor cleanup of platform_wallet_libs

* Indent

* Define RAIBLOCKS_VERSION MAJOR and MINOR only where needed

* Force cmake to store libraries and executables in the root of the build dir

This fixes the tests because they could not find the rai_node executable

* Unify file extension list between check-commit-format and clang-format-all

* Apply clang-format-all.sh

* Change lmdb reference to official repo

And point to the correct branch

* Fix wallet_work_get

* Optional data for amount_impl

* Less lines

* Simplify wallet_balances

* Fix

* Hopefully fix wallet.seed_work_generation

* Fix possible wallet locked errors in RPC (nanocurrency#977)

send, receive, change

* Fix

* Fix peer_container.split

* Remove unnecessary observer subscriptions to reduce a bit of locking overhead

* Explicitly set message type values

* Separate new versions into a new DB table

* Fix nano_wallet

* Fix building rai_node

* Allow the "bulk_pull" bootstrapping message to accept a block hash as… (nanocurrency#973)

Allow the "bulk_pull" bootstrapping message to accept a block hash as its first argument in replace of an account hash

* Update README.md

updated medium link

* Make "bulk_pull" with a start block inclusive of that start block (nanocurrency#985)

* Fixed a test that was relying on incorrect semantics for "bulk_pull"

* Make "bulk_pull" with a start block inclusive of that start block

* Appveyor Fixes (nanocurrency#1009)

* Appveyor Fixes

Fixed Pull Request building
Configured Build matrix for Live and Beta
Updated for MSVC 2017 build environment

* Docker deployment

Stop trying to deploy docker unless in nanocurrency/raiblocks repo

* typo :/

* Update deploy-docker.sh

* Update deploy-docker.sh

remove travis changes, will redo in another branch

* Minor cleanups in cmake code (nanocurrency#1007)

* cmake,style: remove conditions in else() and endif() clauses

It is redundant and makes the code harder to read.

* Group argon2-related declarations

* Group blake2-related declarations

* Move core_test and slow_test to their own CMakeLists.txt

* Set correct interface include directories to gtest target

With this change targets that link to gtest get the correct include
directories automatically.

* Move all nano_wallet declarations together

* Do not define an intermediate variable sources

* Remove duplicated dependency on lmdb

* Converting epoch tag to an enum class instead of a uint8_t.
Fixed an issue where a version was incorrectly being converted to a uint256_t giving incorrect behavior in sucessor clearing.

* Using full typename.

* Futurproofing case statements if a new enumeration value is added.

* Formatting.

* Fix travis builds for OSX (nanocurrency#1016)

* Don't insert elections if we're stopped.

* Moving vote announce loop in to its own thread.

* Formatting.

* Fix

* Move processing of votes in to its own thread.

* Using a single transaction across all votes being processed.

* Use library for deterministic_key

* rai::pub_key function from library

* Prevent possible empty responses

* Eliminate dispatching broadcasting winner from queueing on an IO thread and calculating rep votes in a single transaction instead of multiple.

* Referencing correct parameter.

* Allowing more test iterations.

* Fix epoch block fork process_result

* Disable double vote request for forks

nanocurrency#803
cover this case

* Fix rpc.confirmation_history test (nanocurrency#1027)

* Testing for absense of source block in elections.

* Check if a block can still fit before sending out a vote and if not, terminate the election.

* Fixing merge conflicts and bounding test iterations.

* Formatting.

* Remove transaction write lock from bootstrap

* Consolidate received_frontier transactions

* Empty response message when there's no error code value

* Remove active blocks from *_pending RPC calls

* Add option to include active in RPC *_pending

* Attach epoch to mdb_val since the block store needs to encode this information anyway.
This cleans up a lot of call-site logic to figure out epoch of account_info and pending_info.

* Automated Build for Beta (nanocurrency#1035)

* Log votes for very long unconfirmed elections (nanocurrency#1031)

* Log votes for very long unconfirmed elections

* Fix

* Fix

* Fix

* Minimum announcement minimum cut in half should reduce network usage (nanocurrency#1036)

*  Avoid block_processor queue duplicates (nanocurrency#1000)

* Avoid block_processor duplicates

* Using unordered_set

* blocks.push_back

* forced.push_back

* Send confirmation request to all peers if representatives list is empty… (nanocurrency#999)

* Send confirmation request to all peers if representatives list is empty or observer weight is less than online_weight_minimum

* Fix

* Fix

* More intuitive comparision

* Calculate if representative isn't recorded for several IP addresses

* Use rep_acct

* Save network with many elections in progress

* Increase rep_crawl count if observed peers weight < online_weight_minimum (nanocurrency#1001)

* Increase rep_crawl speed if observed peers weight < online_weight_minimum

* Use uint16_t

* peers.online_weight_minimum

* Improve network::broadcast_confirm_req

* Calculate if representative isn't recorded for several IP addresses

* docker latest tag not applied if regex RC matches (nanocurrency#1042)

f the tag matches RC dont tag as latest in docker (closes nanocurrency#1004)

* Bulk pull account (nanocurrency#1039)

Added "bulk_pull_account" bootstrapping request

*  Allow OpenSSL 1.1 (nanocurrency#950)

Tested with OpenSSL 1.1.0f (Debian Stretch) and OpenSSL 1.0.2g (Ubuntu 16.04 LTS)

* Deadlines in tests (nanocurrency#1026)

* Deadline support for google tests

* Migrated node.cpp to deadlines

* Migrated gap_cache.cpp and ledger.cpp to deadlines

* Migrated network.cpp to deadlines

* Migrated rpc.cpp to deadlines

* Migrated wallet(s).cpp to deadlines

* Migrated qt.cpp to deadlines

* Fix missing return in deadline-test PR (nanocurrency#1049)

* Do not mix confirmed & aborted elections (nanocurrency#1048)

* Do not mix confirmed & aborted elections

* Add delay before aborting election

In case of uncommited write transaction in block_processor

* Fix node.fork_open_flip test

* Fixing possible tests issues

node.fork_multi_flip, node.fork_open, node.fork_flip, node.fork_keep

* Prevent spam from long unconfirmed elections (nanocurrency#1011)

* If an endpoint is not connected, don't throw an exception when logging its IP (nanocurrency#1051)

* In debug mode, rethrow caught exceptions (nanocurrency#1050)

* Replacing RPC wallet_balance_total with wallet_info (nanocurrency#776)

* Replacing RPC wallet_balance_total with wallet_info

Additional info:
- accounts count
- deterministic index
- deterministic count
- adhoc_count

* Add hash only votes, which can be batched (nanocurrency#1025)

* Bump protocol version to Protocol V13

* Fixed failing is_v6() assertions.

* Fix votes.add_cooldown

* Using an explicit operator for conversions from mdb_val instead of a function so names won't drift from typenames.

* Make mdb_val understand account_info instead of the other way around.

* Make mdb_val understand pending_info instead of the other way around.

* Make mdb_val understand pending_key instead of the other way around.

* Make mdb_val understand block_info instead of the other way around.

* Make block_store understand genesis instead of the other way around.

* Moving lmdb stuff to its own file.

* Using explicit type conversion on mdb_val instead of constructor on account_info.

* Using explicit type conversion on mdb_val instead of constructor on pending_info.

* Using explicit type conversion on mdb_val instead of constructor on pending_key.

* Using explicit type conversion on mdb_val instead of constructor on block_info.

* Using explicit type conversion on mdb_val instead of constructor on vote.

* Removing unused function and fix formatting.

* Encapsulating some db-specific implementation-detail functions from the block_store interface.

* Creating store_iterator implementation class so it can be parameterized on different DB backends.

* Adding conversion for uint128_union.

* Adding rai::block operators for mdb_val.

* Formatting.

* Type can be reduced to an std::pair now that the secondary_storage bool was removed.

* Removing unnecessary indirection.

* Allow a sentinel end iterator that doesn't require an implementation iterator.

* formatting

* Using rai::pub_key instead of ed25519_publickey

* Don't use library in secure

* Qt wallet peer count, sorting and width

* Fix rai::store_merge_iterator::cursor_current

* Use DB comparison operator instead of directly using memcmp.

* Rewriting store_merge_iterator in terms of store_iterator for extra reuse.

* Adding is_end_sentinal function to determine if an iterator represents the end iterator.

* Ensure active_transaction thread is stopped on shutdown.

* Convert store_iterator_impl to a generic base class and subclass for both mdb_iterator and mdb_merge_iterator.

* Adding strong typing for store iterators.

* Support non-prefixed block deserialization.

* Correcting types for top-level wallet name iteration.

* Remove exposing mdb_val from the interface of store_iterator_impl.

* Stuff

* Remove file

* Merged in PR#1061, PR#1062, and PR#1072 and associated fixes from master from beta network testing

* Moved time for hash by votes timestamp to 2 weeks from now

* Bump version number to patch release

* Fix possible write transaction lock (nanocurrency#1079)

* Qt editing fix (nanocurrency#1080)

* Initialize rep_crawler_exists (nanocurrency#1084)

* Pass around a transaction inside the wallet (nanocurrency#1083)

* Move enter_initial_password out of node.background (nanocurrency#1081)

* Increment V15 version to V15.2

* Do not vote if voting is disabled in config (nanocurrency#1078)

* Ensure full blocks are not generated by "compute_rep_votes" (nanocurrency#1090)

* Updated rep_weights.bin for bootstrap weights (nanocurrency#1086)

* Remove ded files

* Things missed in merge

* Update blockstore

* Update common

* Update common

* Update utility

* Update secure/ledger

* Update stats

* Update versioning

* Update versioning.hpp

* Update

* Update node

* Update

* Update node

* Update errors

* Fix merge err

* Update node wallet

* Update

* Update bananode entry

* Update

* Fix imports

* Change name

* Un-do

* Fix readme
@rkeene rkeene added this to the V15.0 milestone Oct 18, 2018
@rkeene rkeene added enhancement major This item indicates the need for or supplies a major or notable change labels Oct 18, 2018
Matoking added a commit to Matoking/nanolib that referenced this pull request Apr 30, 2019
Verify epoch V1 state blocks correctly as described in
nanocurrency/nano-node#955
Matoking added a commit to Matoking/nanolib that referenced this pull request Apr 30, 2019
Verify epoch V1 state blocks correctly as described in
nanocurrency/nano-node#955
@argakiig argakiig deleted the epoch-blocks branch May 20, 2019 12:43
Matoking added a commit to Matoking/nanolib that referenced this pull request May 20, 2019
Verify epoch V1 state blocks correctly as described in
nanocurrency/nano-node#955
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement major This item indicates the need for or supplies a major or notable change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants