Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Revert "Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake" #30883

Merged
merged 1 commit into from
Sep 12, 2024

Conversation

hebasto
Copy link
Member

@hebasto hebasto commented Sep 12, 2024

This reverts commit b07fe66 from #30842.

Fixes #30881.

Apparently, the string(APPEND ...) command isn't optimized for large strings.

@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 12, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK maflcko

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@maflcko
Copy link
Member

maflcko commented Sep 12, 2024

review ACK fdeb717

@DrahtBot DrahtBot changed the title Revert "build: Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake" build: Revert "Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake" Sep 12, 2024
@fanquake
Copy link
Member

Going to merge this, given multiple developers have reported issues.

@fanquake fanquake merged commit 07c7c96 into bitcoin:master Sep 12, 2024
5 of 16 checks passed
@hebasto hebasto deleted the 240912-slow-string branch September 12, 2024 16:17
@l0rinc
Copy link
Contributor

l0rinc commented Sep 12, 2024

Fixed in #30888

TheCharlatan added a commit to TheCharlatan/rust-bitcoinkernel that referenced this pull request Sep 16, 2024
…6e67b2a

d8b6e67b2a kernel: Add check if the chainstate maanger is busy loading blocks
f12a131af8 kernel: Process transactions
75d48e6718 kernel: Add optional mempool
889632bb70 kernel: Add support for handling transactions
476c754f3b kernel: Add check for when a block has been mutated.
ebef1b465b kernel: Add support for handling block headers
a56c79a2b5 kernel: Add utxo set iteration and value retrieval
33c71843e3 kernel: Add pure kernel bitcoin-chainstate
ccb2bb59f9 kernel: Add block index utility functions to C header
c953e1f530 kernel: Add function to read block undo data from disk to C header
55673f7f70 kernel: Add functions to read block from disk to C header
348e1c0a22 kernel: Add function for copying  block data to C header
4c71ffbef6 kernel: Add functions for the block validation state to C header
160343c9c2 kernel: Add validation interface and task runner to C header
55a60d8c3f kernel: Add interrupt function to C header
0d59e95b82 kernel: Add import blocks function to C header
804e064ff6 kernel: Add chainstate load options for in-memory dbs in C header
1c829ce9f4 kernel: Add options for reindexing in C header
c8a2917004 kernel: Add block validation to C header
8015b8374c Kernel: Add chainstate loading to kernel C header
b3c8467b62 kernel: Add chainstate manager object to C header
07c55185c2 kernel: Add notifications context option to C header
71c45c0680 kerenl: Add chain params context option to C header
d2ad67bd5f kernel: Add kernel library context object
1c6716c609 kernel: Add logging to kernel library C header
63a83b8dad kernel: Introduce initial kernel C header API
REVERT: 8777c555fc kernel: Add pure kernel bitcoin-chainstate
REVERT: fcc69abdb9 kernel: Add block index utility functions to C header
REVERT: 266d21c9c6 kernel: Add function to read block undo data from disk to C header
REVERT: 62b2926c0a kernel: Add functions to read block from disk to C header
REVERT: 440d7a7a86 kernel: Add function for copying  block data to C header
REVERT: 8827ebd5ea kernel: Add functions for the block validation state to C header
REVERT: 1ad10251b4 kernel: Add validation interface and task runner to C header
REVERT: b96eb0c49c kernel: Add interrupt function to C header
REVERT: d96ffb9165 kernel: Add import blocks function to C header
REVERT: 28dc294a13 kernel: Add chainstate load options for in-memory dbs in C header
REVERT: 7de1202a01 kernel: Add options for reindexing in C header
REVERT: 2e28c5516d kernel: Add block validation to C header
REVERT: f511ce1489 Kernel: Add chainstate loading to kernel C header
REVERT: 7ed0a89a3e kernel: Add chainstate manager object to C header
REVERT: 3fb9cec5d4 kernel: Add notifications context option to C header
REVERT: d71bbbab94 kerenl: Add chain params context option to C header
REVERT: 7cfc892152 kernel: Add kernel library context object
REVERT: ba01b6bd0e kernel: Add logging to kernel library C header
REVERT: 17b98b95eb kernel: Introduce initial kernel C header API
REVERT: 0c4ff18ee9 Merge bitcoin/bitcoin#30896: kernel: Move background load thread to node context
REVERT: 87d54500bf Merge bitcoin/bitcoin#30892: test: Check already deactivated network stays suspended after dumptxoutset
REVERT: 71af7435ef Merge bitcoin/bitcoin#30233: refactor: move m_is_inbound out of CNodeState
REVERT: 1d5b2406bb Merge bitcoin/bitcoin#30877: code style: update .editorconfig file
REVERT: fea550b480 Merge bitcoin/bitcoin#30890: doc: unit test runner help fixup and improvements
REVERT: 95560616fb code style: update .editorconfig file
REVERT: 282f0e9255 Unit test runner documentation fix and improvements
REVERT: 06a9f7789e Merge bitcoin/bitcoin#30433: build: add `standard branch-protection` to hardening flags for aarch64-linux
REVERT: bc7900f33d kernel: Move background load thread to node context
REVERT: e43ce250c6 Merge bitcoin-core/gui#835: Fix crash when closing wallet
REVERT: 001b1cf010 build: use standard branch-protection for aarch64-linux
REVERT: a965f2bc07 gui: fix crash when closing wallet
REVERT: 72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before
REVERT: cf0120ff02 Merge bitcoin/bitcoin#30880: test: Wait for local services to update in feature_assumeutxo
REVERT: e46bebb444 Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
REVERT: be768dbd18 Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage
REVERT: 07c7c96022 Merge bitcoin/bitcoin#30883: build: Revert "Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`"
REVERT: 24817e8b15 Merge bitcoin/bitcoin#30814: kernel: Create usable static kernel library
REVERT: fdeb717e78 Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`"
REVERT: 07f4cebe57 refactor: move m_is_inbound out of CNodeState
REVERT: 19f4a7c95a test: Wait for local services to update in feature_assumeutxo
REVERT: 7d43bca052 Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner
REVERT: cf786eccd7 Merge bitcoin/bitcoin#30865: build: Skip secp256k1 ctime tests when tests are not being built
REVERT: 23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built
REVERT: fa5bc450d5 util: Use compile-time check for LogConnectFailure
REVERT: fa7087b896 util: Use compile-time check for FatalErrorf
REVERT: faa62c0112 util: Add ConstevalFormatString
REVERT: 72b46f28bf test: fix exclude parsing for functional runner
REVERT: a5e99669cc Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration
REVERT: 0c1e507278 Merge bitcoin/bitcoin#30871: build: Add more cmake presets
REVERT: fcb61bbc8d Merge bitcoin/bitcoin#27038: security-check: test for `_FORTIFY_SOURCE` usage in release binaries
REVERT: 85833cf05f Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds
REVERT: 11e2f9fff4 Merge bitcoin/bitcoin#30835: build: Introduce "Kernel" installation component
REVERT: db8350b0e3 Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments
REVERT: a86e7a476d Merge bitcoin/bitcoin#30838: build: Use CMake's default permissions in macOS `deploy` target
REVERT: f0eb63399a Merge bitcoin/bitcoin#30841: ci: Post CMake-migration fixes and amendments
REVERT: 155963768a Merge bitcoin/bitcoin#30842: build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`
REVERT: c773618886 Merge bitcoin/bitcoin#30867: build: Fix `ENABLE_WALLET` option
REVERT: 349632e022 Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD
REVERT: f6298a878f Merge bitcoin/bitcoin#30840: docs: Updated debug build instructions for cmake
REVERT: a8809aeb6e Merge bitcoin/bitcoin#30870: docs: updated developer notes for --with-sanitizers to -DSANITIZERS
REVERT: f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan)
REVERT: fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py
REVERT: 4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags
REVERT: 1eac96a503 Compare FromUserHex result against other hex validators and parsers
REVERT: 19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160
REVERT: 0725a37494 Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions
REVERT: 0037d53d1a build: Fix `ENABLE_WALLET` option
REVERT: 992f83bb6f test: add coverage for assumeUTXO honest peers disconnection
REVERT: 6d5812e5c8 assumeUTXO: fix peers disconnection during sync
REVERT: 082779d606 test: Add explicit onion bind to p2p_permissions
REVERT: c66c68345e Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S()
REVERT: 2756797eca Merge bitcoin/bitcoin#30065: init: fixes file descriptor accounting
REVERT: 5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target
REVERT: e4fb97a512 Merge bitcoin/bitcoin#30791: build: Use correct variable name
REVERT: df3f63ccfa Merge bitcoin/bitcoin#30509: multiprocess: Add -ipcbind option to bitcoin-node
REVERT: 743ac30e34 Add std::optional support to Boost's equality check
REVERT: 712a2b5453 Merge bitcoin/bitcoin#30817: test: Add coverage for dumptxoutset failure robustness
REVERT: fb52023ee6 Merge bitcoin/bitcoin#30684: init: fix init fatal error on invalid negated option value
REVERT: 746f88000e Merge bitcoin/bitcoin#30401: fix: increase consistency of rpcauth parsing
REVERT: 2d68c3b1c2 build: Use correct variables when passing `-fsanitize` to libsecp256k1
REVERT: df86a4f333 Merge bitcoin/bitcoin#30845: Update libsecp256k1 subtree to latest master
REVERT: be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py
REVERT: 94bc3c4cc0 Merge bitcoin/bitcoin#30824: cmake: decouple `FORTIFY_SOURCE` check from `Debug` build type
REVERT: ba84c2774d Merge bitcoin/bitcoin#30823: cmake: add `USE_SOURCE_PERMISSIONS` to all `configure_file()` usage
REVERT: da3f4cb8ee Merge bitcoin/bitcoin#30850: doc: fix minor typo
REVERT: 7a669fde18 docs: Fix minor typo
REVERT: 1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option
REVERT: 0b003e1ff7 docs: Updated debug build instructions for cmake
REVERT: 341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option
REVERT: 5c80192ff6 test: Drop no longer needed workarounds
REVERT: ff54395de4 Update secp256k1 subtree to latest master
REVERT: 611562806c Squashed 'src/secp256k1/' changes from 642c885b61..2f2ccc4695
REVERT: b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`
REVERT: c45186ca54 ci: Switch from `make` to `cmake --build`
REVERT: 6e5f33af58 ci: Handle log files regardless of CMake's version
REVERT: fdad128b52 build: Stop enabling CMake's CMP0141 policy
REVERT: b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions
REVERT: 73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary
REVERT: f03c942095 build, test: Add missed log options
REVERT: 6f2cb0eafd doc: Amend comment about ZeroMQ config files
REVERT: 0dd16d7118 build: Add a pkg-config file for libbitcoinkernel
REVERT: 45be32f838 build: Produce a usable static kernel library
REVERT: 43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests
REVERT: c6c994cb2b test: remove test-only uint160S
REVERT: 62cc4656e2 test: remove test-only uint256S
REVERT: adc00ad728 test: remove test-only arith_uint256S
REVERT: a5fa90706a Merge bitcoin/bitcoin#30834: test: Work around boost compilation error
REVERT: 7b04fabe2d build: Introduce "Kernel" installation component
REVERT: fa9d7d5d20 test: Work around boost compilation error
REVERT: fa3ecdf778 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18"
REVERT: 30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node
REVERT: bbf95c0cc5 Merge bitcoin/bitcoin#30755: ci: Add missed configuration options to "Win64 native" job
REVERT: 73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions
REVERT: 955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods
REVERT: 4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix
REVERT: ee22bf55e3 doc: Update and amend MSVC build guide
REVERT: c07fdd6546 fuzz: Don't compile BDB-specific code on MSVC in `wallet_bdb_parser.cpp`
REVERT: e07a3ede52 ci: Add missed configuration options to "Win64 native" job
REVERT: 1f054eca4e cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage
REVERT: 0e5cd608da Merge bitcoin/bitcoin#30415: contrib: fix check-deps.sh to check for weak symbols
REVERT: 118b55c462 Merge bitcoin/bitcoin#30790: bench: Remove redundant logging benchmarks
REVERT: c0cbe26a86 Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED
REVERT: c3af4b1ec3 Merge bitcoin/bitcoin#30822: cmake: scope Boost Test check to `vcpkg`
REVERT: 7f472e9bcd Merge bitcoin/bitcoin#30821: build: work around issue with Boost <= 1.80 and Clang >= 18
REVERT: d4c7c4009d init: error out if -maxconnections is negative
REVERT: c773649481 init: improves file descriptors accounting and docs
REVERT: 29008a7ff4 init: fixes fd accounting regarding poll/select
REVERT: 30803a35d5 cmake: decouple FORTIFY_SOURCE check from Debug build type
REVERT: a7a4e11db8 cmake: scope Boost Test check to vcpkg
REVERT: d661e2b1b7 Merge bitcoin/bitcoin#30812: lint: Check for release note snippets in the wrong folder
REVERT: cd062d6684 build: work around issue with Boost <= 1.80 and Clang >= 18
REVERT: d6a1b94ffd Merge bitcoin-core/gui#834: qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls
REVERT: 6852d1d487 Merge bitcoin/bitcoin#30796: test: Use std::span and std::string_view for raw data
REVERT: fa3a7ebe5b lint: Check for release note snippets in the wrong folder
REVERT: fa05ee0517 Merge bitcoin/bitcoin#30772: build: Fix / improve coverage scripts
REVERT: 79772cd26e Merge bitcoin/bitcoin#30743: depends: build libevent with `-D_GNU_SOURCE`
REVERT: faecca9a85 test: Use span for raw data
REVERT: c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional
REVERT: 4b5bf335ad test: Add coverage for failing dumptxoutset behavior
REVERT: f794a0d5f4 Merge bitcoin/bitcoin#30819: doc: fix assumeutxo design doc link
REVERT: fadbcd51fc bench: Remove redundant logging benchmarks
REVERT: fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf
REVERT: e5f7272ad3 doc: fix assumeutxo design doc link
REVERT: 93e48240bf Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array
REVERT: f640b323bd Merge bitcoin/bitcoin#30723: lint: Speed up and fix flake8 checks
REVERT: 3ae35b427f ci: run check-deps.sh as part of clang-tidy job
REVERT: 0aaa1298a0 contrib: fix check-deps.sh when libraries do not import symbols
REVERT: 3c99f5a38a contrib: fix check-deps.sh to check for weak symbols
REVERT: 86c80e9cf2 contrib: make check-deps.sh script work with cmake
REVERT: 5373aa30e2 Merge bitcoin/bitcoin#30788: test: fixing failing system_tests/run_command under some Locales
REVERT: 3210d87dfc Merge bitcoin/bitcoin#29043: fuzz: make FuzzedDataProvider usage deterministic
REVERT: 81276540d3 Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli
REVERT: 210210c923 Merge bitcoin/bitcoin#29566: test: update satoshi_round function
REVERT: b0c3de6847 Merge bitcoin/bitcoin#28417: contrib/signet/miner updates
REVERT: cb65ac469a Merge bitcoin/bitcoin#29605: net: Favor peers from addrman over fetching seednodes
REVERT: b8d2f58e06 Merge bitcoin/bitcoin#30808: rpc: dumptxoutset height parameter follow-ups (29553)
REVERT: f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV
REVERT: d9fcbfc372 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script
REVERT: e7cf4a6f27 build: Add missed `-g` for "Coverage" build configuration
REVERT: fe2003ab12 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips
REVERT: a3108a7c56 rpc: Manage dumptxoutset rollback with RAII class
REVERT: c5eaae3b89 doc: Add -rpcclienttimeout=0 to loadtxoutset examples
REVERT: 598b9bba5a rpc: Don't re-enable previously disabled network after dumptxoutset
REVERT: ae48a22a3d test: fixing failing system_tests/run_command under some Locales
REVERT: fac973647d test: Use string_view for json_tests
REVERT: 5567754087 depends: build libevent with -D_GNU_SOURCE
REVERT: 7346b01092 qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls
REVERT: fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT
REVERT: 2ad560139b Remove unused src_dir param from run_tests
REVERT: 2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity
REVERT: fafdb7df34 lint: Speed up flake8 checks
REVERT: faf17df7fb lint: Document missing py_lint dependency
REVERT: faebeb828f lint: Remove python whitespace and shadowing lint rules
REVERT: 7777047835 lint: Remove python lint rules that are SyntaxError
REVERT: faaf3e53f0 test: [refactor] Fix F841 flake8
REVERT: 444421db69 test: [refactor] Fix E714 pycodestyle
REVERT: ee47ca29d6 init: fix fatal error on '-wallet' negated option value
REVERT: 27c976d11a fix: increase consistency of rpcauth parsing
REVERT: 2ad3689512 test: add norpcauth test
REVERT: 67df0dec1a test: blank rpcauth CLI interaction
REVERT: fb6d51eb25 signet/miner: Use argparse exclusive groups
REVERT: ec317bc44b test: update satoshi_round function
REVERT: ecc98ccff2 test: add cases for blank rpcauth
REVERT: c8e6771af0 test: restrict multiple CLI arguments
REVERT: 8838c4f171 common/args.h: automate check for multiple cli commands
REVERT: 6eeb188d40 test: adds seednode functional tests
REVERT: 3270f0adad net: Favor peers from addrman over fetching seednodes
REVERT: 8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array
REVERT: c4762b0aa0 test: allow excluding func test by name and arg
REVERT: 338a266a9a signet/miner: add support for a poolnum/poolid tag in mined blocks
REVERT: 409ab7d35b signet/miner: add Generate.mine function
REVERT: 7b31332370 signet/miner: add Generate.gbt function
REVERT: 85c5c0bea9 signet/miner: add Generate.next_block_time function
REVERT: 5540e6ca49 signet/miner: move next_block_* functions into new Generator class
REVERT: 35f4631196 signet/miner: rename do_decode_psbt to decode_psbt
REVERT: aac040b439 signet/miner: drop create_coinbase function
REVERT: 16951f549e signet/miner: drop do_createpsbt function
REVERT: 3aed0a4284 signet/miner: drop get_reward_address function
REVERT: 01960c53c7 fuzz: make FuzzedDataProvider usage deterministic

git-subtree-dir: libbitcoinkernel-sys/bitcoin
git-subtree-split: d8b6e67b2a12332f188320e90262e2d741e27f7b
TheCharlatan added a commit to TheCharlatan/rust-bitcoinkernel that referenced this pull request Sep 16, 2024
…e7b1f5e

d5ee7b1f5e kernel: Add check if the chainstate maanger is busy loading blocks
604752a1da kernel: Process transactions
37d74344e8 kernel: Add optional mempool
fdfb45e273 kernel: Add support for handling transactions
5ed4aa699c kernel: Add check for when a block has been mutated.
b8f6a2740d kernel: Add support for handling block headers
1aad96a695 kernel: Add utxo set iteration and value retrieval
8777c555fc kernel: Add pure kernel bitcoin-chainstate
fcc69abdb9 kernel: Add block index utility functions to C header
266d21c9c6 kernel: Add function to read block undo data from disk to C header
62b2926c0a kernel: Add functions to read block from disk to C header
440d7a7a86 kernel: Add function for copying  block data to C header
8827ebd5ea kernel: Add functions for the block validation state to C header
1ad10251b4 kernel: Add validation interface and task runner to C header
b96eb0c49c kernel: Add interrupt function to C header
d96ffb9165 kernel: Add import blocks function to C header
28dc294a13 kernel: Add chainstate load options for in-memory dbs in C header
7de1202a01 kernel: Add options for reindexing in C header
2e28c5516d kernel: Add block validation to C header
f511ce1489 Kernel: Add chainstate loading to kernel C header
7ed0a89a3e kernel: Add chainstate manager object to C header
3fb9cec5d4 kernel: Add notifications context option to C header
d71bbbab94 kerenl: Add chain params context option to C header
7cfc892152 kernel: Add kernel library context object
ba01b6bd0e kernel: Add logging to kernel library C header
17b98b95eb kernel: Introduce initial kernel C header API
0c4ff18ee9 Merge bitcoin/bitcoin#30896: kernel: Move background load thread to node context
87d54500bf Merge bitcoin/bitcoin#30892: test: Check already deactivated network stays suspended after dumptxoutset
71af7435ef Merge bitcoin/bitcoin#30233: refactor: move m_is_inbound out of CNodeState
1d5b2406bb Merge bitcoin/bitcoin#30877: code style: update .editorconfig file
fea550b480 Merge bitcoin/bitcoin#30890: doc: unit test runner help fixup and improvements
95560616fb code style: update .editorconfig file
282f0e9255 Unit test runner documentation fix and improvements
06a9f7789e Merge bitcoin/bitcoin#30433: build: add `standard branch-protection` to hardening flags for aarch64-linux
bc7900f33d kernel: Move background load thread to node context
e43ce250c6 Merge bitcoin-core/gui#835: Fix crash when closing wallet
001b1cf010 build: use standard branch-protection for aarch64-linux
a965f2bc07 gui: fix crash when closing wallet
72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before
cf0120ff02 Merge bitcoin/bitcoin#30880: test: Wait for local services to update in feature_assumeutxo
e46bebb444 Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
be768dbd18 Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage
07c7c96022 Merge bitcoin/bitcoin#30883: build: Revert "Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`"
24817e8b15 Merge bitcoin/bitcoin#30814: kernel: Create usable static kernel library
fdeb717e78 Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`"
07f4cebe57 refactor: move m_is_inbound out of CNodeState
19f4a7c95a test: Wait for local services to update in feature_assumeutxo
7d43bca052 Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner
cf786eccd7 Merge bitcoin/bitcoin#30865: build: Skip secp256k1 ctime tests when tests are not being built
23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built
fa5bc450d5 util: Use compile-time check for LogConnectFailure
fa7087b896 util: Use compile-time check for FatalErrorf
faa62c0112 util: Add ConstevalFormatString
72b46f28bf test: fix exclude parsing for functional runner
a5e99669cc Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration
0c1e507278 Merge bitcoin/bitcoin#30871: build: Add more cmake presets
fcb61bbc8d Merge bitcoin/bitcoin#27038: security-check: test for `_FORTIFY_SOURCE` usage in release binaries
85833cf05f Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds
11e2f9fff4 Merge bitcoin/bitcoin#30835: build: Introduce "Kernel" installation component
db8350b0e3 Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments
a86e7a476d Merge bitcoin/bitcoin#30838: build: Use CMake's default permissions in macOS `deploy` target
f0eb63399a Merge bitcoin/bitcoin#30841: ci: Post CMake-migration fixes and amendments
155963768a Merge bitcoin/bitcoin#30842: build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`
c773618886 Merge bitcoin/bitcoin#30867: build: Fix `ENABLE_WALLET` option
349632e022 Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD
f6298a878f Merge bitcoin/bitcoin#30840: docs: Updated debug build instructions for cmake
a8809aeb6e Merge bitcoin/bitcoin#30870: docs: updated developer notes for --with-sanitizers to -DSANITIZERS
f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan)
fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py
4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags
1eac96a503 Compare FromUserHex result against other hex validators and parsers
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160
0725a37494 Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions
0037d53d1a build: Fix `ENABLE_WALLET` option
992f83bb6f test: add coverage for assumeUTXO honest peers disconnection
6d5812e5c8 assumeUTXO: fix peers disconnection during sync
082779d606 test: Add explicit onion bind to p2p_permissions
c66c68345e Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S()
2756797eca Merge bitcoin/bitcoin#30065: init: fixes file descriptor accounting
5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target
e4fb97a512 Merge bitcoin/bitcoin#30791: build: Use correct variable name
df3f63ccfa Merge bitcoin/bitcoin#30509: multiprocess: Add -ipcbind option to bitcoin-node
743ac30e34 Add std::optional support to Boost's equality check
712a2b5453 Merge bitcoin/bitcoin#30817: test: Add coverage for dumptxoutset failure robustness
fb52023ee6 Merge bitcoin/bitcoin#30684: init: fix init fatal error on invalid negated option value
746f88000e Merge bitcoin/bitcoin#30401: fix: increase consistency of rpcauth parsing
2d68c3b1c2 build: Use correct variables when passing `-fsanitize` to libsecp256k1
df86a4f333 Merge bitcoin/bitcoin#30845: Update libsecp256k1 subtree to latest master
be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py
94bc3c4cc0 Merge bitcoin/bitcoin#30824: cmake: decouple `FORTIFY_SOURCE` check from `Debug` build type
ba84c2774d Merge bitcoin/bitcoin#30823: cmake: add `USE_SOURCE_PERMISSIONS` to all `configure_file()` usage
da3f4cb8ee Merge bitcoin/bitcoin#30850: doc: fix minor typo
7a669fde18 docs: Fix minor typo
1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option
0b003e1ff7 docs: Updated debug build instructions for cmake
341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option
5c80192ff6 test: Drop no longer needed workarounds
ff54395de4 Update secp256k1 subtree to latest master
611562806c Squashed 'src/secp256k1/' changes from 642c885b61..2f2ccc4695
b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`
c45186ca54 ci: Switch from `make` to `cmake --build`
6e5f33af58 ci: Handle log files regardless of CMake's version
fdad128b52 build: Stop enabling CMake's CMP0141 policy
b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions
73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary
f03c942095 build, test: Add missed log options
6f2cb0eafd doc: Amend comment about ZeroMQ config files
0dd16d7118 build: Add a pkg-config file for libbitcoinkernel
45be32f838 build: Produce a usable static kernel library
43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests
c6c994cb2b test: remove test-only uint160S
62cc4656e2 test: remove test-only uint256S
adc00ad728 test: remove test-only arith_uint256S
a5fa90706a Merge bitcoin/bitcoin#30834: test: Work around boost compilation error
7b04fabe2d build: Introduce "Kernel" installation component
fa9d7d5d20 test: Work around boost compilation error
fa3ecdf778 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18"
30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node
bbf95c0cc5 Merge bitcoin/bitcoin#30755: ci: Add missed configuration options to "Win64 native" job
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions
955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods
4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix
ee22bf55e3 doc: Update and amend MSVC build guide
c07fdd6546 fuzz: Don't compile BDB-specific code on MSVC in `wallet_bdb_parser.cpp`
e07a3ede52 ci: Add missed configuration options to "Win64 native" job
1f054eca4e cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage
0e5cd608da Merge bitcoin/bitcoin#30415: contrib: fix check-deps.sh to check for weak symbols
118b55c462 Merge bitcoin/bitcoin#30790: bench: Remove redundant logging benchmarks
c0cbe26a86 Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED
c3af4b1ec3 Merge bitcoin/bitcoin#30822: cmake: scope Boost Test check to `vcpkg`
7f472e9bcd Merge bitcoin/bitcoin#30821: build: work around issue with Boost <= 1.80 and Clang >= 18
d4c7c4009d init: error out if -maxconnections is negative
c773649481 init: improves file descriptors accounting and docs
29008a7ff4 init: fixes fd accounting regarding poll/select
30803a35d5 cmake: decouple FORTIFY_SOURCE check from Debug build type
a7a4e11db8 cmake: scope Boost Test check to vcpkg
d661e2b1b7 Merge bitcoin/bitcoin#30812: lint: Check for release note snippets in the wrong folder
cd062d6684 build: work around issue with Boost <= 1.80 and Clang >= 18
d6a1b94ffd Merge bitcoin-core/gui#834: qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls
6852d1d487 Merge bitcoin/bitcoin#30796: test: Use std::span and std::string_view for raw data
fa3a7ebe5b lint: Check for release note snippets in the wrong folder
fa05ee0517 Merge bitcoin/bitcoin#30772: build: Fix / improve coverage scripts
79772cd26e Merge bitcoin/bitcoin#30743: depends: build libevent with `-D_GNU_SOURCE`
faecca9a85 test: Use span for raw data
c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional
4b5bf335ad test: Add coverage for failing dumptxoutset behavior
f794a0d5f4 Merge bitcoin/bitcoin#30819: doc: fix assumeutxo design doc link
fadbcd51fc bench: Remove redundant logging benchmarks
fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf
e5f7272ad3 doc: fix assumeutxo design doc link
93e48240bf Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array
f640b323bd Merge bitcoin/bitcoin#30723: lint: Speed up and fix flake8 checks
3ae35b427f ci: run check-deps.sh as part of clang-tidy job
0aaa1298a0 contrib: fix check-deps.sh when libraries do not import symbols
3c99f5a38a contrib: fix check-deps.sh to check for weak symbols
86c80e9cf2 contrib: make check-deps.sh script work with cmake
5373aa30e2 Merge bitcoin/bitcoin#30788: test: fixing failing system_tests/run_command under some Locales
3210d87dfc Merge bitcoin/bitcoin#29043: fuzz: make FuzzedDataProvider usage deterministic
81276540d3 Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli
210210c923 Merge bitcoin/bitcoin#29566: test: update satoshi_round function
b0c3de6847 Merge bitcoin/bitcoin#28417: contrib/signet/miner updates
cb65ac469a Merge bitcoin/bitcoin#29605: net: Favor peers from addrman over fetching seednodes
b8d2f58e06 Merge bitcoin/bitcoin#30808: rpc: dumptxoutset height parameter follow-ups (29553)
f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV
d9fcbfc372 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script
e7cf4a6f27 build: Add missed `-g` for "Coverage" build configuration
fe2003ab12 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips
a3108a7c56 rpc: Manage dumptxoutset rollback with RAII class
c5eaae3b89 doc: Add -rpcclienttimeout=0 to loadtxoutset examples
598b9bba5a rpc: Don't re-enable previously disabled network after dumptxoutset
ae48a22a3d test: fixing failing system_tests/run_command under some Locales
fac973647d test: Use string_view for json_tests
5567754087 depends: build libevent with -D_GNU_SOURCE
7346b01092 qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls
fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT
2ad560139b Remove unused src_dir param from run_tests
2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity
fafdb7df34 lint: Speed up flake8 checks
faf17df7fb lint: Document missing py_lint dependency
faebeb828f lint: Remove python whitespace and shadowing lint rules
7777047835 lint: Remove python lint rules that are SyntaxError
faaf3e53f0 test: [refactor] Fix F841 flake8
444421db69 test: [refactor] Fix E714 pycodestyle
ee47ca29d6 init: fix fatal error on '-wallet' negated option value
27c976d11a fix: increase consistency of rpcauth parsing
2ad3689512 test: add norpcauth test
67df0dec1a test: blank rpcauth CLI interaction
fb6d51eb25 signet/miner: Use argparse exclusive groups
ec317bc44b test: update satoshi_round function
ecc98ccff2 test: add cases for blank rpcauth
c8e6771af0 test: restrict multiple CLI arguments
8838c4f171 common/args.h: automate check for multiple cli commands
6eeb188d40 test: adds seednode functional tests
3270f0adad net: Favor peers from addrman over fetching seednodes
8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array
c4762b0aa0 test: allow excluding func test by name and arg
338a266a9a signet/miner: add support for a poolnum/poolid tag in mined blocks
409ab7d35b signet/miner: add Generate.mine function
7b31332370 signet/miner: add Generate.gbt function
85c5c0bea9 signet/miner: add Generate.next_block_time function
5540e6ca49 signet/miner: move next_block_* functions into new Generator class
35f4631196 signet/miner: rename do_decode_psbt to decode_psbt
aac040b439 signet/miner: drop create_coinbase function
16951f549e signet/miner: drop do_createpsbt function
3aed0a4284 signet/miner: drop get_reward_address function
01960c53c7 fuzz: make FuzzedDataProvider usage deterministic
REVERT: d8b6e67b2a kernel: Add check if the chainstate maanger is busy loading blocks
REVERT: f12a131af8 kernel: Process transactions
REVERT: 75d48e6718 kernel: Add optional mempool
REVERT: 889632bb70 kernel: Add support for handling transactions
REVERT: 476c754f3b kernel: Add check for when a block has been mutated.
REVERT: ebef1b465b kernel: Add support for handling block headers
REVERT: a56c79a2b5 kernel: Add utxo set iteration and value retrieval
REVERT: 33c71843e3 kernel: Add pure kernel bitcoin-chainstate
REVERT: ccb2bb59f9 kernel: Add block index utility functions to C header
REVERT: c953e1f530 kernel: Add function to read block undo data from disk to C header
REVERT: 55673f7f70 kernel: Add functions to read block from disk to C header
REVERT: 348e1c0a22 kernel: Add function for copying  block data to C header
REVERT: 4c71ffbef6 kernel: Add functions for the block validation state to C header
REVERT: 160343c9c2 kernel: Add validation interface and task runner to C header
REVERT: 55a60d8c3f kernel: Add interrupt function to C header
REVERT: 0d59e95b82 kernel: Add import blocks function to C header
REVERT: 804e064ff6 kernel: Add chainstate load options for in-memory dbs in C header
REVERT: 1c829ce9f4 kernel: Add options for reindexing in C header
REVERT: c8a2917004 kernel: Add block validation to C header
REVERT: 8015b8374c Kernel: Add chainstate loading to kernel C header
REVERT: b3c8467b62 kernel: Add chainstate manager object to C header
REVERT: 07c55185c2 kernel: Add notifications context option to C header
REVERT: 71c45c0680 kerenl: Add chain params context option to C header
REVERT: d2ad67bd5f kernel: Add kernel library context object
REVERT: 1c6716c609 kernel: Add logging to kernel library C header
REVERT: 63a83b8dad kernel: Introduce initial kernel C header API

git-subtree-dir: libbitcoinkernel-sys/bitcoin
git-subtree-split: d5ee7b1f5ee19e454a74d00502a0bf7be849c260
fanquake added a commit that referenced this pull request Sep 17, 2024
…,Json}.cmake

2a58114 build: Minimize I/O operations in GenerateHeaderFromJson.cmake (Lőrinc)
aa003d1 build: Minimize I/O operations in GenerateHeaderFromRaw.cmake (Lőrinc)

Pull request description:

  Follow up of the #30883 revert.

  Replaced multiple file writes with a single string template write.
  The raw content is first grouped into 8 byte chunks, followed by another regex replace which wraps them in `std::byte` or just the raw bytes, prefixed with `0x`.

  Tested the output with `diff -w` and they're the same - only whitespace differences because slightly different source formatting.

  ----

  Tested the `Raw` performance with:
  ```bash
  time cmake -DRAW_SOURCE_PATH=src/bench/data/block413567.raw -DHEADER_PATH=build/after/block413567.raw.h -DRAW_NAMESPACE=benchmark::data -P cmake/script/GenerateHeaderFromRaw.cmake
  ```

  Before:
  > 15.41s user 23.06s system 97% cpu 39.593 total

  After:
  > 0.77s user 0.06s system 97% cpu 0.849 total

  ----

  Tested the `Json` performance with:
  ```bash
  time cmake -DJSON_SOURCE_PATH=src/secp256k1/src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.json -DHEADER_PATH=build/after/ecdsa_secp256k1_sha256_bitcoin_test.json -P cmake/script/GenerateHeaderFromJson.cmake
  ````

  Before:
  > 3.57s user 6.01s system 94% cpu 10.136 total

  After:
  > 0.17s user 0.01s system 98% cpu 0.187 total

ACKs for top commit:
  maflcko:
    review ACK 2a58114 👒
  hebasto:
    ACK 2a58114.
  willcl-ark:
    tACK 2a58114

Tree-SHA512: 5e44f79d1c0dbb61d8b64f28d4c3c87a176981f72104b28800eef2037b0728076cbcf14ff07b05ff94d4e8800605586cfd5df00519db9027933c5943348c01d2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cmake: GenerateHeaderFrom very slow
5 participants