feat(COP): only in testnet: reduce the number of block producers #9563

nikurt · 2023-09-22T10:30:26Z

In chains that are not mainnet:

Decrease the number of block producers to 20
Decrease the number of chunk only producers to 100

This change lets testnet in its current state perform testing of chunk-only producers.

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

wacban · 2023-09-22T10:51:25Z

core/primitives/src/epoch_manager.rs

+        use_production_config: bool,
+        genesis_epoch_config: EpochConfig,
+        chain_id: &str,


Not related but it would be cool to remove use_production_config as a follow up. I don't remember why but it really annoyed me.
Generally I love the idea of customizing EpochConfigs based on the chain. I don't know if the chain name is the best way to go about it but it is an improvement.

Good idea, but better to do it in a separate PR.

wacban · 2023-09-22T10:51:29Z

core/primitives/src/epoch_manager.rs

+        {
+            let num_shards = config.shard_layout.num_shards() as usize;
+            // On testnet, genesis config set num_block_producer_seats to 200
+            // This is to bring it back to 100 to be the same as on mainnet


not 💯anymore

This change will decrease the total number of validators as well. Maybe increase the number of cop to keep it high enough so that more people can join in as validators? I don't feel strongly about it either way, just brainstorming.

not 100 anymore

fixed

will decrease the total number of validators

Indeed, but with the current number of testnet validators it doesn't seem to be an issue.
Will increase the number of chunk producers if somebody has a strong opinion in favor of that.

@jakmeier

Chunk header `gas_limit` was renamed to `prev_gas_limit` in #9500. After discussion with @jakmeier I've realised while it is determined in the previous chunk, it actually corresponds to gas limit for the current chunk.

The clone was only there because of lifetime complaints that were due to the message being moved to the test observer after processing. We can avoid the clone entirely in non-test setup. This is important because these messages can be 100MB+

Continues #9540 Deletes the `DelayDetector` class and replaces the uses of it with spans. --------- Co-authored-by: Anton Puhach <anton@near.org> Co-authored-by: Simonas Kazlauskas <git@kazlauskas.me> Co-authored-by: Jakob Meier <mail@jakobmeier.ch>

RoutingTableView contains a table Direct Peers with one entry per peer. The table displays the latest distance vector shared by each peer. This PR: - Adds the Min Nonce for each distance vector to the display, indicating the age of the received information. - Moves any peers in a disconnected state to a separate table for clarity. <img width="1448" alt="image" src="https://github.com/near/nearcore/assets/3241341/8488eb07-7adf-4c7c-838e-165ca740d2fa">

Versions 0.5.1 - 0.5.7 were yanked due to a race condition. See [changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-channel/CHANGELOG.md#version-058) and [crates.io](https://crates.io/crates/crossbeam-channel/versions) and [PR with the fix](crossbeam-rs/crossbeam#972).

…manager, runtime in integration tests (#9595) We had a common pattern of creating a bunch of manual stores, epoch managers, and nightshade runtime for the test environment. This PR brings the default functionality to the test environment builder.

The point of this file is to quickly see the current parameters. Showing the nightly changes is not in this spirit IMO.

`cargo-deny advisories` complains about an old libsqlite-sys version. We depend on it through rusqlite, which is only used in the `estimator-warehouse`, a CI /&debug script. So it's not super important but still worth updating.

Breaking a large test_utils.rs file into multiple parts. Additionally shifting location of `TestEnvNightshadeSetupExt` to nearcore so that it can be used in other modules, specifically `tools/state-viewer/src/state_dump.rs`. Note that we can't have `TestEnvNightshadeSetupExt` in `chain/client/src/test_utils` as it depends on NightshadeRuntime which is a part of `nearcore` module. `nearcore` module depends on `chain` and would create a cyclic dependency.

…t` (#9608) I got absolutely fed up with waiting for prerequisite infrastructure work for #9290 (why does it have to be _that_ hard?) However the PR in question had some other important improvements that do not necessarily *rely* on changes to said infrastructure to work. In particular: * `nextest` now retries failing tests a few times before giving up, to make sure they aren't spuriously failing; * This should help some timing out integration tests in particular, as those often fail because of some deadlockish situation in my experience. * style checks still run with `cargo nextest` – unfortunately that means that in CI this will run these checks multiple times, but that doesn’t sound particularly terrible of a tradeoff (especially if the other changes mean we won't be retrying entire test suites as often anymore;) * This should allow for a greater portion of the test suite to run on Macs – unfortunately not verified by the CI, but people do complain and this should make the situation better. cc #9367

This brings down the runtime of tests involving the runtime-tester by 40%, ie. 7.5s -> 4.5s for the runtime quickcheck added in #9602. Hopefully other tests will also profit from it, though I haven’t benchmarked it all. The overall runtime on buildkite of the `cargo test nightly not integration-tests*` test seems, on a sample of 1, to be roughly the same (+- 10s) before addition of the test from #9602, so there should not be a significant delay in CI round-trip due to it, and it should mostly impact local development that rarely sees changes from dependencies.

Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Shreyan Gupta <shreyan@near.org>

It seems we can get this minor update without issues. Which is nice because 0.10.55 included a fix to [RUSTSEC-2023-0044](https://rustsec.org/advisories/RUSTSEC-2023-0044). Not something that affects as AFAIK but better to fix it nevertheless.

The state-dump command changes the amounts of validator accounts to match the epoch we're dumping state in, in case a validator stakes during that epoch, and the locked amount is different than the stake. The new locked amount computation is correct if a validator staked more, but overflows if the locked amount is smaller than the stake plus amount, which actually does occur on mainnet

) Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Shreyan Gupta <shreyan@near.org> Co-authored-by: Simonas Kazlauskas <git@kazlauskas.me> Co-authored-by: Ekleog-NEAR <96595974+Ekleog-NEAR@users.noreply.github.com>

After trying out a couple of things to try to get the unit tests working with snapshotting for our resharding module, I feel like the complexity of resharding functions has reached a place where it might be fine to just remove the unit test module in favor of the integration tests. We have more flexibility with the integration tests and for now, the unit test and the integration tests are basically doing the same thing, The effort to make the unit test work with snapshot is probably not worth it.

… a precise block (#9598) With the current implementation for flat storage iterator, we have the issue that we can not control the block over which we create the iterator. A solution to that is to use the snapshot mechanism from state sync. We can assume that the snapshot would be created on all nodes as of the last block of an epoch. The way we get the correct state for resharding is that we take the snapshot as of the `prev_prev_hash` and append the delta from the `prev_hash` so that the state of the child tries/shards matches that of the last block of prev epoch.

Implements near/NEPs#492

…9630) Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.13 to 1.26.17. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/releases">urllib3's releases</a>.</em></p> <blockquote> <h2>1.26.17</h2> <ul> <li>Added the <code>Cookie</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>. (GHSA-v845-jxx5-vc9f)</li> </ul> <h2>1.26.16</h2> <ul> <li>Fixed thread-safety issue where accessing a <code>PoolManager</code> with many distinct origins would cause connection pools to be closed while requests are in progress (<a href="https://redirect.github.com/urllib3/urllib3/issues/2954">#2954</a>)</li> </ul> <h2>1.26.15</h2> <ul> <li>Fix socket timeout value when HTTPConnection is reused (<a href="https://redirect.github.com/urllib3/urllib3/issues/2645">urllib3/urllib3#2645</a>)</li> <li>Remove "!" character from the unreserved characters in IPv6 Zone ID parsing (<a href="https://redirect.github.com/urllib3/urllib3/issues/2899">urllib3/urllib3#2899</a>)</li> <li>Fix IDNA handling of 'x80' byte (<a href="https://redirect.github.com/urllib3/urllib3/issues/2901">urllib3/urllib3#2901</a>)</li> </ul> <h2>1.26.14</h2> <ul> <li>Fixed parsing of port 0 (zero) returning None, instead of 0 (<a href="https://redirect.github.com/urllib3/urllib3/issues/2850">#2850</a>)</li> <li>Removed deprecated <code>HTTPResponse.getheaders()</code> calls in <code>urllib3.contrib</code> module.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's changelog</a>.</em></p> <blockquote> <h1>1.26.17 (2023-10-02)</h1> <ul> <li>Added the <code>Cookie</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>. (<code>[#3139](urllib3/urllib3#3139) <https://github.com/urllib3/urllib3/pull/3139></code>_)</li> </ul> <h1>1.26.16 (2023-05-23)</h1> <ul> <li>Fixed thread-safety issue where accessing a <code>PoolManager</code> with many distinct origins would cause connection pools to be closed while requests are in progress (<code>[#2954](urllib3/urllib3#2954) <https://github.com/urllib3/urllib3/pull/2954></code>_)</li> </ul> <h1>1.26.15 (2023-03-10)</h1> <ul> <li>Fix socket timeout value when <code>HTTPConnection</code> is reused (<code>[#2645](urllib3/urllib3#2645) <https://github.com/urllib3/urllib3/issues/2645></code>__)</li> <li>Remove "!" character from the unreserved characters in IPv6 Zone ID parsing (<code>[#2899](urllib3/urllib3#2899) <https://github.com/urllib3/urllib3/issues/2899></code>__)</li> <li>Fix IDNA handling of '\x80' byte (<code>[#2901](urllib3/urllib3#2901) <https://github.com/urllib3/urllib3/issues/2901></code>__)</li> </ul> <h1>1.26.14 (2023-01-11)</h1> <ul> <li>Fixed parsing of port 0 (zero) returning None, instead of 0. (<code>[#2850](urllib3/urllib3#2850) <https://github.com/urllib3/urllib3/issues/2850></code>__)</li> <li>Removed deprecated getheaders() calls in contrib module. Fixed the type hint of <code>PoolKey.key_retries</code> by adding <code>bool</code> to the union. (<code>[#2865](urllib3/urllib3#2865) <https://github.com/urllib3/urllib3/issues/2865></code>__)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/urllib3/urllib3/commit/c9016bf464751a02b7e46f8b86504f47d4238784"><code>c9016bf</code></a> Release 1.26.17</li> <li><a href="https://github.com/urllib3/urllib3/commit/01220354d389cd05474713f8c982d05c9b17aafb"><code>0122035</code></a> Backport GHSA-v845-jxx5-vc9f (<a href="https://redirect.github.com/urllib3/urllib3/issues/3139">#3139</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/e63989f97d206e839ab9170c8a76e3e097cc60e8"><code>e63989f</code></a> Fix installing <code>brotli</code> extra on Python 2.7</li> <li><a href="https://github.com/urllib3/urllib3/commit/2e7a24d08713a0131f0b3c7197889466d645cc49"><code>2e7a24d</code></a> [1.26] Configure OS for RTD to fix building docs</li> <li><a href="https://github.com/urllib3/urllib3/commit/57181d6ea910ac7cb2ff83345d9e5e0eb816a0d0"><code>57181d6</code></a> [1.26] Improve error message when calling urllib3.request() (<a href="https://redirect.github.com/urllib3/urllib3/issues/3058">#3058</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/3c0148048a523325819377b23fc67f8d46afc3aa"><code>3c01480</code></a> [1.26] Run coverage even with failed jobs</li> <li><a href="https://github.com/urllib3/urllib3/commit/d94029b7e2193ff47b627906a70e06377a09aae8"><code>d94029b</code></a> Release 1.26.16</li> <li><a href="https://github.com/urllib3/urllib3/commit/18e92145e9cddbabdf51c98f54202aa37fd5d4c8"><code>18e9214</code></a> Use trusted publishing for PyPI</li> <li><a href="https://github.com/urllib3/urllib3/commit/d25cf83bbae850a290fe34ed1610ae55c0558b36"><code>d25cf83</code></a> [1.26] Fix invalid test_ssl_failure_midway_through_conn</li> <li><a href="https://github.com/urllib3/urllib3/commit/25cca389496b86ee809c21e5b641aeaa74809263"><code>25cca38</code></a> [1.26] Fix test_ssl_object_attributes</li> <li>Additional commits viewable in <a href="https://github.com/urllib3/urllib3/compare/1.26.13...1.26.17">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=1.26.13&new-version=1.26.17)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/near/nearcore/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: wacban <wacban@users.noreply.github.com> Co-authored-by: Anton Puhach <anton@near.org> Co-authored-by: Simonas Kazlauskas <git@kazlauskas.me> Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Saketh Are <saketh.are@gmail.com> Co-authored-by: Shreyan Gupta <shreyan@near.org> Co-authored-by: Ekleog-NEAR <96595974+Ekleog-NEAR@users.noreply.github.com> Co-authored-by: Marcelo Diop-Gonzalez <marcelo827@gmail.com> Co-authored-by: Bowen Wang <bowenwang1996@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

This PR adds chunk production with post-state-root. This means applying transactions and receipts before producing the chunk and including the necessary fields in the header. The code is not used anywhere yet, that will be implemented as a separate PR. The issues marked with `TODO(post-state-root)` will be resolved separately. Part of #9450.

Addressing https://github.com/near/nearcore/blob/65af455f6f7f1bb8bc8cb399666468daebb28c66/core/primitives/src/errors.rs#L86

This is a replacement of #9516 , with no unsafe code (well there is, but it's safe). All of the trie nodes sit in an Arena, which is clean and safe, and easy to clean up.

…9632) Pure refactor, better locality of functions for making changes/adding new functions related to snapshots.

@jakmeier

…atus`, `broadcast_tx_commit`, `broadcast_tx_async` (#9556) This PR is a part of RPC tx methods redesign https://docs.google.com/document/d/1jGuXzBlMAGQENsUpy5x5Xd7huCLOd8k6tyMOEE41Rro/edit?usp=sharing Addressing point 3 in #9542 and start working on #6837 Changes: - New field `status` appeared in the response of RPC methods `tx`, `EXPERIMENTAL_tx_status`, `broadcast_tx_commit`, `broadcast_tx_async` - Added some comments with the explanations In #6837, Illia suggested to have the structure ```rust enum BroadcastWait { /// Returns right away. None, /// Waits only for inclusion into the block. Inclusion, /// Waits until block with inclusion is finalized. InclusionFinal, /// Waits until all non-refund receipts are executed. Executed, /// Waits until all non-refund receipts are executed and the last of the blocks is final. ExecutedFinal, /// Waits until everything has executed and is final. Final, } ``` I've renamed it to `TxExecutionStatus` and simplified it a little by dropping all refund options: 1. We may add them later without breaking backwards compatibility 2. To support them, we need to have the access to `receipt` (it's the method `get_tx_execution_status`). We have there only `execution_outcome` by default. We created special logic to be able not to retrieve the receipts (it's the only difference between `tx` and `EXPERIMENTAL_tx_status`). I have a feeling (please correct me here if I'm wrong) that retrieving corresponding receipt may slow down the execution that we tried to optimise. BTW, maybe the data from `execution_outcome` is enough (we have there `gas_burnt` and `tokens_burnt`). @jakmeier suggested the condition `outcome.tokens_burnt == 0 && outcome.status == Success`. Unfortunately, we need to work with any status. But maybe the first part (`outcome.tokens_burnt == 0`) is enough to say that it could be only refund receipt. I need more input here.

This PR is a part of RPC tx methods redesign https://docs.google.com/document/d/1jGuXzBlMAGQENsUpy5x5Xd7huCLOd8k6tyMOEE41Rro/edit?usp=sharing It has no usages on our RPC nodes, but I can't say anything about 3rd party nodes. It is not covered by documentation in any of our sources. If there's anyone who uses this method, I suggest switching to `tx` method instead. The return type differs, giving more information to the user.

wacban

LGTM
I'm fine with adding it to nightly. For stabilization I would feel a bit better if one more person approved as well.

wacban · 2023-10-11T13:16:52Z

core/primitives/src/epoch_manager.rs

+        if chain_id != "mainnet"
+            && checked_feature!("stable", TestnetFewerBlockProducers, protocol_version)


This will include mocknet.

Yes, and that is intentional. I'd like to test this change in mocknet.

Hmm, okay but I am a bit weary about diverging mocknet and mainnet.

gmilescu

Looks good.

wacban · 2023-10-11T14:56:28Z

core/primitives/src/epoch_manager.rs

+        genesis_epoch_config: EpochConfig,
+        chain_id: &str,
+    ) -> Self {
+        Self { use_production_config, genesis_epoch_config, chain_id: chain_id.to_string() }
    }

    pub fn for_protocol_version(&self, protocol_version: ProtocolVersion) -> EpochConfig {


When testing on mocknet please remember to set use_production_config to true, otherwise your logic won't be triggered - see two lines below.

Above you suggested to remove use_production_config. That should take care of mocknet configuration

Sure, as long as you do that first, you'll be fine. It will make my life easier as well when testing resharding.

wacban · 2023-10-11T14:57:20Z

core/primitives/src/epoch_manager.rs

+        if chain_id != "mainnet"
+            && checked_feature!("stable", TestnetFewerBlockProducers, protocol_version)


Hmm, okay but I am a bit weary about diverging mocknet and mainnet.

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

Implements near/NEPs#492

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

Breaking a large test_utils.rs file into multiple parts. Additionally shifting location of `TestEnvNightshadeSetupExt` to nearcore so that it can be used in other modules, specifically `tools/state-viewer/src/state_dump.rs`. Note that we can't have `TestEnvNightshadeSetupExt` in `chain/client/src/test_utils` as it depends on NightshadeRuntime which is a part of `nearcore` module. `nearcore` module depends on `chain` and would create a cyclic dependency.

… a precise block (#9598) With the current implementation for flat storage iterator, we have the issue that we can not control the block over which we create the iterator. A solution to that is to use the snapshot mechanism from state sync. We can assume that the snapshot would be created on all nodes as of the last block of an epoch. The way we get the correct state for resharding is that we take the snapshot as of the `prev_prev_hash` and append the delta from the `prev_hash` so that the state of the child tries/shards matches that of the last block of prev epoch.

GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.

I mixed up adding tests and introducing new functionality here #9648 Please have a look just at tests separately

[borsh-rs 1.0.0 release](https://github.com/near/borsh-rs/releases/tag/borsh-v1.0.0) is a major milestone and it makes sense to update nearcore to use it. Here is the migration guide that was captured along the way: https://github.com/near/borsh-rs/blob/borsh-v1.0.0/docs/migration_guides/v0.10.2_to_v1.0.0_nearcore.md

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

In chains that are not `mainnet`: * Decrease the number of block producers to 20 * Decrease the number of chunk only producers to 100 This change lets testnet in its current state perform testing of chunk-only producers. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Anton Puhach <anton@near.org> Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Simonas Kazlauskas <git@kazlauskas.me> Co-authored-by: Saketh Are <saketh.are@gmail.com> Co-authored-by: Shreyan Gupta <shreyan@near.org> Co-authored-by: Ekleog-NEAR <96595974+Ekleog-NEAR@users.noreply.github.com> Co-authored-by: Marcelo Diop-Gonzalez <marcelo827@gmail.com> Co-authored-by: Bowen Wang <bowenwang1996@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: wacban <wacban@users.noreply.github.com> Co-authored-by: Adam Chudaś <18039094+staffik@users.noreply.github.com> Co-authored-by: robin-near <111538878+robin-near@users.noreply.github.com> Co-authored-by: Olga Telezhnaya <olyatelezhnaya@gmail.com> Co-authored-by: i-fix-typos <146758284+i-fix-typos@users.noreply.github.com> Co-authored-by: Aleksandr Logunov <alex.logunov@near.org> Co-authored-by: Jure Bajic <jure@near.org> Co-authored-by: dj8yf0μl <26653921+dj8yfo@users.noreply.github.com>

frol · 2023-11-01T18:52:10Z

core/primitives/src/epoch_manager.rs

+        if chain_id != crate::chains::MAINNET
+            && checked_feature!("stable", TestnetFewerBlockProducers, protocol_version)
+        {
+            let num_shards = config.shard_layout.num_shards() as usize;
+            // Decrease the number of block producers from 100 to 20.
+            config.num_block_producer_seats = 20;
+            config.num_block_producer_seats_per_shard =
+                vec![config.num_block_producer_seats; num_shards];
+            // Decrease the number of chunk producers.
+            config.validator_selection_config.num_chunk_only_producer_seats = 100;
+        }


How will it affect Calimero? cc @chefsale @miraclx

I would consider adding one more check before overriding genesis config: the date of creation of the network. Thus, newly created networks would respect the value from the genesis config.

OR maybe target specifically "testnet" chain id instead of "not mainnet"

feat(COP): Increase the number of chunk-only producers in testnet

fa266eb

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

wacban reviewed Sep 22, 2023

View reviewed changes

pugachAG and others added 28 commits October 11, 2023 14:10

refactor: undo gas_limit renaming (#9558)

ce230e1

Chunk header `gas_limit` was renamed to `prev_gas_limit` in #9500. After discussion with @jakmeier I've realised while it is determined in the previous chunk, it actually corresponds to gas limit for the current chunk.

fix: remove nightly changes from parameters.snap (#9612)

97d5859

The point of this file is to quickly see the current parameters. Showing the nightly changes is not in this spirit IMO.

chore: bump rusqlite version (#9615)

cb6a1f0

`cargo-deny advisories` complains about an old libsqlite-sys version. We depend on it through rusqlite, which is only used in the `estimator-warehouse`, a CI /&debug script. So it's not super important but still worth updating.

refactor(state-sync): Rename module: syncing -> state_sync (#9594)

cf4589a

Co-authored-by: Jakob Meier <mail@jakobmeier.ch> Co-authored-by: Shreyan Gupta <shreyan@near.org>

chore: bump openssl version (#9614)

d2d28b4

It seems we can get this minor update without issues. Which is nice because 0.10.55 included a fix to [RUSTSEC-2023-0044](https://rustsec.org/advisories/RUSTSEC-2023-0044). Not something that affects as AFAIK but better to fix it nevertheless.

feat(runtime): restrict creation of non-implicit TLA (#9589)

a18c96f

Implements near/NEPs#492

[resharding] Enable snapshot iterator for resharding (#9607)

a611048

Add hash of the node to the MissingTrieValue (#9616)

9192147

Addressing https://github.com/near/nearcore/blob/65af455f6f7f1bb8bc8cb399666468daebb28c66/core/primitives/src/errors.rs#L86

[memtrie] (1/n) Encoding and decoding of in-memory trie nodes. (#9541)

2a7d676

This is a replacement of #9516 , with no unsafe code (well there is, but it's safe). All of the trie nodes sit in an Arena, which is clean and safe, and easy to clean up.

[refactor] Move ShardTries snapshot related code to a separate file (#…

bc6acc0

…9632) Pure refactor, better locality of functions for making changes/adding new functions related to snapshots.

doc: fix typos in CONTRIBUTING.md (#9624)

0993de3

nikurt requested a review from Longarithm October 11, 2023 13:10

wacban approved these changes Oct 11, 2023

View reviewed changes

nikurt changed the title ~~feat(COP): Increase the number of chunk-only producers in testnet~~ feat(COP): only in testnet: reduce the number of block producers Oct 11, 2023

nikurt requested review from wacban and gmilescu October 11, 2023 13:29

gmilescu approved these changes Oct 11, 2023

View reviewed changes

wacban reviewed Oct 11, 2023

View reviewed changes

nikurt and others added 16 commits October 11, 2023 18:18

refactor: Introduce constants for commonly used chain names (#9666)

f696e63

feat(COP): Increase the number of chunk-only producers in testnet

f94e2dc

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

feat(runtime): restrict creation of non-implicit TLA (#9589)

877f113

Implements near/NEPs#492

feat(COP): Increase the number of chunk-only producers in testnet

09f7c6a

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

feat(COP): Increase the number of chunk-only producers in testnet

5682fd5

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

merge

e403572

feat(COP): Increase the number of chunk-only producers in testnet

dee2cc6

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

jsonrpc: add tx serialisation tests (#9653)

f419514

I mixed up adding tests and introducing new functionality here #9648 Please have a look just at tests separately

feat(COP): Increase the number of chunk-only producers in testnet

f92c9a5

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

feat(COP): Increase the number of chunk-only producers in testnet

f37ce4f

Only in chain_id=testnet: * Limit the number of block producers to 20 * keep num_chunk_only_producer_seats the same.

merge

c37e310

Merge branch 'master' into nikurt-more-cop

dfd0420

nikurt enabled auto-merge October 11, 2023 16:32

nikurt added this pull request to the merge queue Oct 11, 2023

Merged via the queue into master with commit 6f324e8 Oct 11, 2023
10 checks passed

nikurt deleted the nikurt-more-cop branch October 11, 2023 17:16

frol reviewed Nov 1, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(COP): only in testnet: reduce the number of block producers #9563

feat(COP): only in testnet: reduce the number of block producers #9563

nikurt commented Sep 22, 2023 •

edited

Loading

wacban Sep 22, 2023

nikurt Sep 22, 2023

wacban Sep 22, 2023

nikurt Sep 22, 2023

wacban left a comment

wacban Oct 11, 2023

nikurt Oct 11, 2023

wacban Oct 11, 2023

gmilescu left a comment

wacban Oct 11, 2023

nikurt Oct 11, 2023

wacban Oct 11, 2023

wacban Oct 11, 2023

frol Nov 1, 2023

frol Nov 1, 2023 •

edited

Loading

		if chain_id != "mainnet"
		&& checked_feature!("stable", TestnetFewerBlockProducers, protocol_version)

feat(COP): only in testnet: reduce the number of block producers #9563

feat(COP): only in testnet: reduce the number of block producers #9563

Conversation

nikurt commented Sep 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wacban left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmilescu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frol Nov 1, 2023 • edited Loading

Choose a reason for hiding this comment

nikurt commented Sep 22, 2023 •

edited

Loading

frol Nov 1, 2023 •

edited

Loading