Logging improvements #779
Merged
Logging improvements #779
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cryptocode
force-pushed
the
log-improvements
branch
from
4d33c01 to
e9edb0a
Compare
cryptocode
force-pushed
the
log-improvements
branch
3 times, most recently
from
d8e5233 to
2efee97
Compare
cryptocode
force-pushed
the
log-improvements
branch
from
2efee97 to
9b7cdae
Compare
FCherepanov
added a commit
to artem-kruglov/raiblocks
that referenced
this pull request
Jul 23, 2018
* Spring cleanup of root directory (nanocurrency#784) * Lies. * Only push nano-ci docker image on master Also disables attempting to cache the nano images, because --cache-from can't work with two stage builds, and the default cache will speed up the second beta build. * Remove disabled debug command (nanocurrency#788) * Automatically vacuum database if it gets too large in the Docker container (nanocurrency#792) * Remove some unused captures and locals, and constructor initialization list reordering (nanocurrency#786) * Work generation for state blocks in process RPC (nanocurrency#785) * Optionally disable local work when work peers are down (nanocurrency#791) * Log ledger rollbacks votes * Bool for faster new_winner calculation * typo uint vs unsigned int for msvc * Log each block votes weight * Using a single transaction to check all wallets and checking for account existence in the same transaction. * Adding optional confirm_req answers If tthis is request for forked block/account representative knows about * Using ledger.successor to retrieve correct block * Remove asserts in successor function For debug mode: This is fine to not have successor * Remove not required condition for confirm_req for confirm_req options * Add a query to see if votes are uncontested. * Protocol version constants (nanocurrency#805) * Queueing rollbacks through the block processor instead of in an IO thread. process_receive_many can block on the database lock and might cause IO thread starvation. * Formatting. * Cleaning up and simplifying fork resolution process. * Holding reference to node by weak_ptr so it doesn't suspend the shutdown process. * Pushing blocks to the front of the queue because: 1) During bootstrap it will be received frontier-backwards to newer blocks are less likely to land in unchecked 2) Rollbacks will be resolved quicker * Setting minimum protocol version to network version 7 and depeering any node lower than that. * logging updates (nanocurrency#807) * logging updated Moved logging for unknown block type under network_packet loggin * logging updated various logging messages have been moved into the appropriate config switches * Formatting * Toning down connection spam in log. * Fix OSX build on Travis * Streamline bootstrap logging to remove non-useful information. * Adding logging to block processing queue so we can see it's doing something. * Removing dead code. * Adding priority queue so blocks voted to be rolled back get removed quicker. * Removing block_processor_item type and just using a block shared pointer since forced items are in a separate queue now. * Bumping version numbers. * Windows Specific Logging (nanocurrency#811) Silence Windows Specific Common Errors while bootstrapping * Make elections when bootstrapping more resilient (nanocurrency#810) This does a two things for elections while bootstrapping: - Space out the initial confirm_reqs to reduce packet loss - Retry confirm_reqs for big reps if we don't get a response This also makes rep_crawler more aggressive, so that we have more reps to confirm_req. * Add work and signature to account_history response if raw is true * Fix potential segfault in announce_votes * Disallow account variations passing checksum test (nanocurrency#797) * Fix state block send callback amount to not be inverted (nanocurrency#796) * Logging improvements (nanocurrency#779) * Add balance option for blocks_info RPC action (nanocurrency#794) * Minor RPC block_create changes (nanocurrency#804) * Fetching account balance & previous for send blocks if aren't given directly * Account field is not required for state blocks as it has private/public key * If there is no link in request use source or destination * Double check current balance if previous block is specified * Link can be 0 for changing representative * Simplifying filter loop, correctly handling removal of last item, and not skipping check on item swapped in from the back. * Don't call broadcast_reps with empty list. * Bumping developmen/master version number. * Fixing RPC block_create test (nanocurrency#814) "previous" field can be auto-filled now * Use default values on success/error to give accurate errors instead of ptree exceptions when tests fail (nanocurrency#815) * Correct mdb table comments with Doxygen conversion (nanocurrency#816) * Encapsulating bootstrap_initiator internal data. * Don't hold bootstrap attempt open while forks are being resolved. * Using a low-overhead timeout mechanism using only an atomic int instead of deadline_timer which can involve a syscall/read. * Allow zero balance in block_create for state block sends (nanocurrency#817) * Fixed 'account' field for state send subtype in 'account_history' RPC call (nanocurrency#819) * Fix formatting. * QT 5.9.5 @rpath is set correctly so we don't need to install_name_tool every framework. * Remove unused function and captures (nanocurrency#822) * ledger::supply is a hotspot, so we memoize and update periodically. (nanocurrency#824) * Adding manual block confirmation function. * Adding block_confirm RPC. * Moving block callback post-confirmation. * Fixing formatting. * Merge branch 'sentinal_mode-2' into sentinel-squash * Eliminating duplicate quorum check code. * Checking if resolved at call site instead of within callback. * Statistics support (nanocurrency#759) * Logging quorom weight received. * Fix handling of invalid wallet work (nanocurrency#832) * Convert dynamic casts to static casts (nanocurrency#831) * Add RPC block_hash action (nanocurrency#830) * Add logging to help track down a possible race (nanocurrency#837) * Resolve breaking changes in Boost 1.67 (nanocurrency#829) * Simplify rollback visitor for send blocks (nanocurrency#838) * Expanding the window in which blocks are rebroadcast to the network. Changing from 60 to 300 second retention and also tracking the last 8000 blocks. * Removing private addresses from live network peers https://tools.ietf.org/html/rfc1918 https://tools.ietf.org/html/rfc6598 https://tools.ietf.org/html/rfc4193 * Rewrite unsynced (fixes bulk_push_client) * Add logging * Temporarily disable bulk_pull_logging check for bulk push logging * Revert "Temporarily disable bulk_pull_logging check for bulk push logging" This reverts commit 26b1823. * Fixing merge artifact. * Prevent log deletion by inactive_node (nanocurrency#802) * Prevent log deletion by inactive_node if logging.max_size > default value * Clang formatting * Fix typo keeping bootstraps open during exit (nanocurrency#842) * Push missing blocks back to the peer we requested frontiers from since this is the node missing the blocks. (nanocurrency#844) * Fix inverted check in scan_receivable causing spurious asserts (nanocurrency#846) * Rep votes don't need to be computed during election construction since they're always computed before broadcasting the winner. (nanocurrency#849) Removing unused transaction parameters in election and active_transactions. * Extracting root_exists function to check if the root block or account exists. * Supporting nano_ and xrb_ prefixes. (nanocurrency#854) * Rebroadcast blocks via process RPC (nanocurrency#852) * The process RPC was reimplementing block processing but didn't properly rebroadcast blocks it processed. This converts the RPC to just pass the block off to the block processor. The processing result can be retrieved via logging. * Fixing formatting. * Supporting nano_ and xrb_ prefixes. (nanocurrency#854) (reverted from commit ec7432b) * Supporting nano_ and xrb_ prefixes. Deprecating support for v1 addresses. (nanocurrency#854) (reverted from commit ec7432b) (reverted from commit ae23b77) * Renaming list_sqrt to list_fanout so the naming isn't tied to one form of rebroadcast fanout sizing. Changing data structure to a deque since we don't need the entries to be contiguous. * Halving network fanout since the previous value was extremely conservative. * Simplifying send_keepalive test which won't compile on some STL implementations due to an issue with deque iterators. * Fixed formatting. * Don't put key in wallet otherwise it's a race condition between block_create and the wallet internally receiving the block. * Using a less-than operator instead of greater-than and using vote_info instead of a tuple. * Moving vote_replay code in to vote_processor as a more appropriate place to hang it. Only returning vote code instead of vote used since we don't need to track the vote used and might not always have access to it. Vote replay code now works appropriately if there's an active election for the block. * Continuing to announce votes until quorum is met or the block can no longer be inserted. * Adding logging if blocks aren't being confirmed for an excessive amount of time. * Fixing formatting. * Bumping protocol version on master since we have a V13 stabilization branch. * Fix stateblock receive account field to be consistent with legacy blocks (nanocurrency#820) * Fix stateblock receive account field to be consistent with legacy blocks * Add account field to output of account_history RPC call * Only look up account if it is not provided in account_history RPC call * Statistics pane under Advanced in Qt wallet (nanocurrency#864) * optional use_peers param to generate_work rpc (nanocurrency#862) * Queuing a wallet action in a background thread is unnecessary. (nanocurrency#876) * Simplify and improve performance of search_pending. (nanocurrency#866) * Simplify and improve performance of search_pending. Invert loop iteration since the number of accounts in the wallet is likely smaller than the number of pending entries. Automatic block confirmation already checks if the destination is in a local wallet so we don't need to re-implement this. Re-searching the pending table for source blocks for the confirmed account can be better implemented in the future with an account dirty flag or queued block insertion. Since we don't need a callback after confirmation we don't need the search code in its own class. * Using block_confirm instead of doing it manually. * Fix Appveyor (nanocurrency#875) * Changing frontier_count to account_count (nanocurrency#879) * Count the accounts table instead of frontiers table to fix frontier_count RPC * Renaming frontier_count to account_count. Retaining old frontier_count RPC for compatibility. * Making callback outside of work_pool queue lock. * Fix wallet send action so it respects generate_work parameter and also fix test so to pass false to generate work. * Run work cancel tests in a loop since it's a race condition to successful cancellation. * Using queue_wallet_action for ensure working is generated for newly inserted keys. * Use a common scheme for function naming and name some as blocking to indicate they block while work is being generated. * Removing work_fetch as it's only used in one place. * Fixing formatting. * Update rpc.cpp generate_work was renamed, one instance was missed * Add DNS resolution support for work peers (nanocurrency#865) * Fix typo: quorom -> quorum (nanocurrency#881) * Fix type, quorom -> quorum * Upgrade config JSON * Fix syntax error * Fix formatting * Don't insert a block in to the processing queue if it's already marked as active which implies it's already been added to the queue. * Update node.cpp Fixed nanocurrency#861/nanocurrency#881 as the config version was incremented but not saved to config properly * Only send one confirm_req instead of both. Since v11 nodes have responded with the block they have for the root associated with the block so both aren't needed. * Rename Doxygen name to Nano (nanocurrency#889) * Moving vote deserialization on to the vote class. * Initialize state member in block_counts constructor Fixes nanocurrency#900 * Update ledger stats (nanocurrency#902) * Remove unused transaction from confirmation_history (nanocurrency#903) * Option to force fork resolution in RPC process (nanocurrency#892) * Option to force fork resolution in RPC process * Enabled control should be required for manual fork resolution * Deleting forced block from elections * Add stats for outgoing bulk push (nanocurrency#905) * Making verification a member of the vote class. * Hoisting calls to balance visitor up to reduce recursion. * Update common.cpp clang formatting fix * Calculating tally only once. * Changing local block and vote before confirmation is triggered. * Removing unused arguments. * Fixing formatting. * Resolving forks as part of the node instead of part of the bootstrap process. * Fix formatting. * Disable compute_rep_votes in config if enable_voting set to false, wallet shouldn't compute its account votes * Fork test sometimes needs more iterations. * Stateblock Canary Check Cleanup Stateblocks are now being parsed and generated, Canary blocks no longer matter since only nodes that understand them are still on the network * Removing more canary block code. * Remove unused process_result related code (nanocurrency#911) * Throttling pull requests if blocks can't be processed quick enough which was causing large amounts of memory usage in the block_processor. * Removing excess logging. * Bootstrap connection cleanup (nanocurrency#912) * Deserialize in to receive buffer without any pointer arithmetic. * Creating rai socket class that has timeout capability encapsulated. * Putting message header in its own class and removing buffer pointer arithmetic when parsing bootstrap messages. * Converting bootstrap_server connection to a rai::socket to ensure timeouts are used when sending data. * Version bump Bumping protocol version on master since we have a V14 stabilization branch * Trim whitespaces in certain input fields in Qt wallet to avoid confusion on paste * Adding public keys for release signing * Voting winner patch (by Lee Bousfield) (nanocurrency#916) * Voting winner patch (by Lee Bousfield) * Missed from patch * Revert "Voting winner patch (by Lee Bousfield) (nanocurrency#916)" This reverts commit e1b9c1f. * shared_ptr <rai::block> comparison as hash * CLI unchecked_clear (nanocurrency#915) and optional unchecked_clear for "vacuum" & "snapshot" * Update utility.cpp (nanocurrency#924) support lower end devices that can't handle the existing code. * Simplify search_pending (nanocurrency#922) Voting for each pending block separately * Correct account fields in QT wallet (nanocurrency#914) * Assign nodes a persistent ID * bootstrap improvements Stale forks should be able to be evicted through bootstrapping, even if they've recently come off the network. * Update node.cpp clean up verbose logging to std::cerr by default * Fixing upgrade path for node_id logging in the config * Don't count state blocks twice in stats (nanocurrency#927) * Accurate rep count in logs when sending confirm req (nanocurrency#926) * RPC wallet_change_seed functionality should match CLI (nanocurrency#925) Backports nanocurrency#637. * Set permissions on backup files so only owner can read and update the backup (nanocurrency#929) * GTEST for MSVC 15.5 compatibility * Restart of Beta network (nanocurrency#936) * New beta network genesis * Adding beta network preconfigured reps * Update beta.json Correct RPC port for default beta config.json * Replace call operator with notify function (nanocurrency#940) * Add missing version_put (nanocurrency#939) fixes nanocurrency#919 * Update live.json & beta.json (nanocurrency#938) for Docker images * State blocks output for QT wallet "Create Block" (nanocurrency#934) * State blocks output for QT wallet "Create Block" * Clang formatting * Move boost_log_setup before boost_regex when linking (nanocurrency#933) Without this when compiling rai_node on fedora 28 I get many undefined references starting with: ``` [ 68%] Linking CXX executable rai_node /usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/libboost_log_setup.a(init_from_settings.o): Funktiossa ”boost::log::v2s_mt_posix::(anonymous namespace)::default_syslog_sink_factory<char>::create_sink(boost::log::v2s_mt_posix::basic_settings_section<char> const&)”: (.text+0x1b72): määrittelemätön viite kohteeseen ”boost::log::v2s_mt_posix::sinks::syslog_backend::set_severity_mapper(boost::log::v2s_mt_posix::aux::light_function<boost::log::v2s_mt_posix::sinks::syslog::level (boost::log::v2s_mt_posix::record_view const&)> const&)” ``` * Confirm req to all peers if reps list is empty (nanocurrency#937) or online stake is less than required quorum * Update README.md (nanocurrency#941) Road-map and Twitter Link updates * std::error_code and std::expected implementation * Remove store_entry class since it's effectively just a pair of rai::mdb_val * Speed up travis builds: enable ccache, cache cargo and reduce test timeouts (nanocurrency#957) * Do not hang trying to upload to docker in unofficial repos docker push hung asking for a password if the password environment variable was not set. With this change it fails immediately. * Enable ccache on linux and osx * Cache homebrew downloads This does not avoid the compilation of the homebrew packages but stores the downloaded sources, speeding up the installation process. * Enable cargo caching * Cache load-tester cargo binaries * Reduce default timeout per test suite to 120 seconds One of the tests hangs in the CI system, reducing the timeout to have quicker feedback. It can be re-adjusted in the future if 120 is not enough for a successful run. * Fix logging config serialization * Raw history changes (nanocurrency#943) * Return "balance" in raw history for state blocks * Return "previous" in raw history * Remove previous for open block * Added "bootstrap_connections_max" to beta.json (nanocurrency#960) * CLI refactoring * Fix the work value of the beta network genesis block (nanocurrency#963) * Fix deterministic wallet keys in boost 1.67 or above (nanocurrency#967) * Fix deterministic wallet keys in boost 1.67 or above * Match existing casting idiom * Fix several core_test issues (nanocurrency#959) * Fix test rpc.version * Sometimes election can start before block_processor commit * Attempt to process confirmed block if it's not in ledger yet * Fix QT block creation tests * Enable tests upgrade_v1_v2 & send_single_many_peers * Increase iterations for fork_pre_confirm * Fix -Wconversion-null warnings * Disable confirm_if_quorum for already confirmed election * Fix error with 0 key in wallet store * Redo "Disable confirm_if_quorum" * Increase possible synchronizing time for QT wallet * Log ID for each RPC call (nanocurrency#951) Now it's logged only on completion * Fix more core tests * Fix rpc.wallet_ledger * Fix more time comparison in rpc tests * Disable node.bootstrap_connection_scaling * Fix message.confirm_ack_serialization * Initialize booleans when not explicitly set (nanocurrency#971) * Add epoch blocks to disable old type blocks (nanocurrency#955) * Add version field to account_info and min_version to pending_info * Add epoch block support to ledger * Add epoch block settings to config, and remove state canaries * Add test for epoch blocks * Add receive upgrade test for epoch blocks * Ignore old nodes on the beta network * Add min_version to RPC pending and wallet_pending * Add account_version field to account_info RPC * Fix formatting * Remove epoch block config (de)serializing * Mark epoch blocks in history (nanocurrency#968) * Mark epoch blocks in history * Extra info for raw * Remove epoch block from non-raw RPC history * Move raw check * Fix upgrade_v11_to_v12 when already upgraded * Fix rpc.version test * Fix system.generate_send_existing * Require core tests to pass in Travis (nanocurrency#969) * Fix rai::block_store::upgrade_v11_to_v12 * modify type error: age => count (nanocurrency#932) * Hopefully fix rpc.work_get * Move rai/node library definition to separate CMakeLists.txt * Move rai/lib library definition to separate CMakeLists.txt * Add latest version FindBoost.cmake * Move rai_node to a separate CMakeLists.txt * Move secure into its own folder * Move ed25519 to its own CMakeLists.txt * Avoid compiling rai_lib twice This was apparently done to simplify the linking of the tests but it is not needed if the library is compiled as STATIC as it seems to be currently. This is because CMAKE_SHARED_LIBS is set to off by default. * Fix compilation of core_test * Fix miniupnpc include directories from the main CMakeLists.txt * Fix compilation of GUI * Move badly placed icon from secure to rai_wallet * Fix includes * Fix dependencies * Add missing dependencies to lmdb * Fix compilation of GUI and tests * Another pass removing left-overs in CMakeLists.txt * Set exe linker flags globally * Fix linux compilation: link against threading and dl * Move all GUI specific code to the same if * Only link lmbd dependencies to lmdb * Move ACTIVE_NETWORK definition after option declaration It was being used before its declaration * Restore compiler extensions (fixes compilation on windows) * Use custom lmdb * Fix dependency of wallet on WinExtras and include * Fix compilation and packaging in appveyor * Build install target to build everything necessary for packaging * Download vc_redist before building because the install target requires it * Guard compilation of qt_test under RAIBLOCKS_TEST * Start commenting out extra link libraries, include directories, etc. * Group GUI declarations * Add missing dependency secure -> boost * Start modernizing cmake code * Fix compilation of node * Group all gtest code under if (RAIBLOCKS_TEST) * Remove left-overs in CMakeLists.txt * Comment out platform libs, to be added later if needed minor cleanup of platform_wallet_libs * Indent * Define RAIBLOCKS_VERSION MAJOR and MINOR only where needed * Force cmake to store libraries and executables in the root of the build dir This fixes the tests because they could not find the rai_node executable * Unify file extension list between check-commit-format and clang-format-all * Apply clang-format-all.sh * Change lmdb reference to official repo And point to the correct branch * Fix wallet_work_get * Hopefully fix wallet.seed_work_generation * Fix possible wallet locked errors in RPC (nanocurrency#977) send, receive, change * Fix peer_container.split * Remove unnecessary observer subscriptions to reduce a bit of locking overhead
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BOOST_LOG: