Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json #8423

Merged
merged 6 commits into from
Feb 1, 2023

Conversation

ppca
Copy link
Contributor

@ppca ppca commented Jan 23, 2023

This is useful for dynamic config.
For this to work for node start, nearup also needs to be updated.

@ppca ppca requested a review from nikurt January 23, 2023 23:35
Copy link
Contributor

@nikurt nikurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also allow comments in node_key.json, i.e. https://github.com/near/nearcore/blob/master/nearcore/src/config.rs#L1293 ?
Should we also allow comments in validator_key.json, i.e. https://github.com/near/nearcore/blob/master/core/primitives/src/validator_signer.rs#L163 ?

nearup

I don't think that updating nearup is needed. nearup is not the recommended way of running nodes. If comments are not allowed in those conditions, that is acceptable.

@@ -271,7 +272,9 @@ impl GenesisConfig {
pub fn from_file<P: AsRef<Path>>(path: P) -> anyhow::Result<Self> {
let file = File::open(path).with_context(|| "Could not open genesis config file.")?;
let reader = BufReader::new(file);
let genesis_config: GenesisConfig = serde_json::from_reader(reader)
// Strip the comments from the input (use `as_bytes()` to get a `Read`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Strip the comments from the input (use `as_bytes()` to get a `Read`).
// Strip the comments from the input.
// `as_bytes()` returns a `Read`.

tracing::info!(target: "neard", config=?config, "Changing the config {path:?}.");
return Ok(Some(config));
Ok(config_str) => {
// Strip the comments from the input (use `as_bytes()` to get a `Read`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

@ppca ppca changed the title Use json_comments crate to skip config file comments feat: allow comments in config.json, node_key.json and validator.json Jan 31, 2023
@ppca ppca changed the title feat: allow comments in config.json, node_key.json and validator.json feat: allow comments in config.json, genesis.json, node_key.json and validator.json Jan 31, 2023
@ppca
Copy link
Contributor Author

ppca commented Jan 31, 2023

Should we also allow comments in node_key.json, validator_key.json
Added.

I've tested for dynamic config field expected_shutdown in localnet, the following comments are allowed:
Screenshot 2023-01-30 at 6 20 24 PM

The strip comments for the other json's should also work but not sure how we can test them?

@ppca ppca marked this pull request as ready for review January 31, 2023 02:21
@ppca ppca requested a review from a team as a code owner January 31, 2023 02:21
@nikurt
Copy link
Contributor

nikurt commented Jan 31, 2023

The strip comments for the other json's should also work but not sure how we can test them?

Add comments to node_key.json and validator_key.json. If the node starts then comments are indeed stripped.

@@ -261,18 +261,25 @@ impl GenesisConfig {
///
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment that it's not JSON string, but JSON-with-comments string.

@@ -395,11 +395,12 @@ impl Default for Config {

impl Config {
pub fn from_file(path: &Path) -> anyhow::Result<Self> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment that the file can be JSON-with-comments.

@@ -0,0 +1,11 @@
[package]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this in the global Cargo.toml as a workspace member. Around line 60.


use json_comments::StripComments;

pub fn strip_comments_from_json_str(json_str: &String) -> std::io::Result<String> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea to move all the code that deals with json_comments to a separate module.
👍

@ppca ppca force-pushed the xiangyiz/ND-271 branch 2 times, most recently from 8ef8e2d to 693e452 Compare February 1, 2023 16:11
@ppca ppca merged commit fd4b385 into near:master Feb 1, 2023
@ppca ppca deleted the xiangyiz/ND-271 branch February 1, 2023 17:31
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validator.json feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json feat: allow comments in config.json, genesis.json, node_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json feat: allow comments in config.json, genesis.json, node_key.json and validate_key.json Feb 1, 2023
@ppca ppca changed the title feat: allow comments in config.json, genesis.json, node_key.json and validate_key.json feat: allow comments in config.json, genesis.json, node_key.json and validator_key.json Feb 1, 2023
nikurt pushed a commit to nikurt/nearcore that referenced this pull request Feb 3, 2023
…validator.json (near#8423)

* Use json-comments-rs crate to skip config file comments

* strip comments out from json for genesis_config.json, config.json, validator_key.json and node_key.json

* strip comments functions sit in near-config-utils (utils/config)
nikurt pushed a commit to nikurt/nearcore that referenced this pull request Feb 3, 2023
…validator.json (near#8423)

* Use json-comments-rs crate to skip config file comments

* strip comments out from json for genesis_config.json, config.json, validator_key.json and node_key.json

* strip comments functions sit in near-config-utils (utils/config)
nikurt pushed a commit that referenced this pull request Feb 6, 2023
…validator.json (#8423)

* Use json-comments-rs crate to skip config file comments

* strip comments out from json for genesis_config.json, config.json, validator_key.json and node_key.json

* strip comments functions sit in near-config-utils (utils/config)
@iho
Copy link
Contributor

iho commented Jul 11, 2023

What motivation for having comments in JSON? Can I remove this feature? With this feature, we have higher startup time.

#9127 (comment)

@nikurt
Copy link
Contributor

nikurt commented Jul 12, 2023

The PR description could have been more explicit about it. It greatly speeds up development of features involving config changes.
No, please don't remove this feature.
Responded on the linked comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants