Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ feat(all): add application-wide configuration manager #892

Merged
merged 18 commits into from
Oct 16, 2024

Conversation

raxhvl
Copy link
Contributor

@raxhvl raxhvl commented Oct 14, 2024

πŸ—’οΈ Description

This PR introduces a package to manage environment and application configurations.

.
β”œβ”€β”€ src
β”‚   └── πŸ“ config  [Application wide environment and configurations]
β”‚       β”œβ”€β”€ πŸ“„ __init__.py
β”‚       β”œβ”€β”€ πŸ“„ app.py [Configurations for application framework]
β”‚       β”œβ”€β”€ πŸ“„ docs.py [Configurations for documentation]
β”‚       └── πŸ“„ env.py  [Exposes `env.yaml` to the application]
└── πŸ“„ env.yaml [Environment file (git ignored)]

Environment Configurations

Application-wide environment configuration, which varies across staging, production, and development environments are read from env.yaml in the project root.

This file will not be tracked by git, making it safe for storing local secrets.

To get started, run the command env_init cli to initialize your environment configuration.

Usage

1. Generate env file

Run the env_init cli tool.

➜ uv run env_init
Env file created:  execution-spec-tests/env.yaml

which should generate an env.yaml in the project root.

remote_nodes:
  - name: mainnet_archive
    # Replace with your Ethereum RPC node URL
    node_url: http://example.com
    # Optional: Headers for RPC requests
    rpc_headers:
      client-secret: <secret>

2. Import EnvConfig

from config import EnvConfig
EnvConfig().remote_nodes[0].name
'mainnet_archive'

Application configuration

Application configuration are pydantic classes.

from config import DocsConfig
DocsConfig().TARGET_FORK
'Prague'

πŸ”— Related Issues

closes #891, closes #893

βœ… Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

@raxhvl
Copy link
Contributor Author

raxhvl commented Oct 14, 2024

@danceratopz let me know if this needs a changelog entry.

Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestions thanks! I made a further suggestion to help people get started with config below.

As we discussed via DM, we should probably move away from a JSON config at some point. Unless we go with a pydantic-settings setup for gentest's config, but that might be overkill :)

src/cli/gentest.py Outdated Show resolved Hide resolved
src/cli/gentest.py Outdated Show resolved Hide resolved
config.json.example Outdated Show resolved Hide resolved
src/cli/gentest.py Show resolved Hide resolved
src/cli/gentest.py Outdated Show resolved Hide resolved
@raxhvl raxhvl changed the title ✨ feat(gentest): Allow custom headers and example config ✨ feat(gentest): Better gentest configuration Oct 14, 2024
@raxhvl raxhvl marked this pull request as draft October 15, 2024 06:09
@raxhvl raxhvl changed the title ✨ feat(gentest): Better gentest configuration ✨ feat: Environment configurations Oct 15, 2024
@raxhvl raxhvl marked this pull request as ready for review October 15, 2024 08:10
@raxhvl
Copy link
Contributor Author

raxhvl commented Oct 15, 2024

@danceratopz Let me know where should I document this so the team is aware.

@danceratopz danceratopz added scope:gentest Scope: gentest CLI type:feat type: Feature labels Oct 15, 2024
Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, really like how this is shaping up! β˜„οΈ A few nits below!

src/config/env.py Outdated Show resolved Hide resolved
src/config/env.py Outdated Show resolved Hide resolved
src/config/env.py Outdated Show resolved Hide resolved
src/config/env.py Outdated Show resolved Hide resolved
src/config/env.py Outdated Show resolved Hide resolved
src/config/env.py Show resolved Hide resolved
@danceratopz
Copy link
Member

danceratopz commented Oct 15, 2024

@danceratopz Let me know where should I document this so the team is aware.

As this config will be limited to gentest (for starters), you could re-use gentest's docstrings in a new HTML page for gentest under here:
https://ethereum.github.io/execution-spec-tests/main/library/cli/
which gets generated from:
https://github.com/ethereum/execution-spec-tests/blob/619e7d8340802f457f5f4e1ada0bff3826a1c27c/src/cli/evm_bytes.py
and here:
https://github.com/ethereum/execution-spec-tests/blob/619e7d8340802f457f5f4e1ada0bff3826a1c27c/docs/library/cli/evm_bytes.md

But feel free to leave this until to you start work on gentest proper, it's ok for me if we skip it for now.

If you're very motivated, you could add a short explainer under the developer docs:
https://ethereum.github.io/execution-spec-tests/main/dev/

@raxhvl
Copy link
Contributor Author

raxhvl commented Oct 15, 2024

I have made the following updates

  1. Default configs: Now using a template. Also Default config is itself a Config model so it will also be validated

    DEFAULT_CONFIG = Config(
    remote_nodes=[
    RemoteNode(
    name="mainnet_archive",
    node_url="http://example.com",
    rpc_headers={"client-secret": "<secret>"},
    )
    ]
    )

  2. Overriding exiting environment: Show a friendly message

❯ uv run env_init
🚧 The configuration file 'execution-spec-tests/env.yaml' already exists. Please update it manually if needed.
  1. Nudge user if the env.yaml does not exist
FileNotFoundError: The configuration file 'execution-spec-tests/env.yaml' does not exist. Run `uv run env_int` to create it.

If this looks okay then I will start documenting it.

Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the following updates
....
If this looks okay then I will start documenting it.

Thanks for addressing the nits. Looks great, thanks! Ready for doc.

config.json.example Outdated Show resolved Hide resolved
src/config/templates/env.yaml.j2 Outdated Show resolved Hide resolved
@raxhvl
Copy link
Contributor Author

raxhvl commented Oct 15, 2024

@danceratopz All done.

@raxhvl raxhvl changed the title ✨ feat: Environment configurations ✨ feat: Configurations Oct 16, 2024
@danceratopz danceratopz changed the title ✨ feat: Configurations ✨ feat(all): add application-wide configuration manager Oct 16, 2024
Copy link
Member

@danceratopz danceratopz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! 🌟 Thanks a lot for this!

@danceratopz danceratopz merged commit 3c7e745 into ethereum:main Oct 16, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:gentest Scope: gentest CLI type:feat type: Feature
Projects
None yet
2 participants