-
Notifications
You must be signed in to change notification settings - Fork 448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add: extension env-injector #4925
Conversation
3d7336d
to
e23fac0
Compare
28e698c
to
1f9db53
Compare
b4853bb
to
c823394
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I've tested and everything works as expected.
I have a few concerns, on the testing, error handling, and documentation.
tests/spread/extensions/snaps/env-injector-hello/snap/snapcraft.yaml
Outdated
Show resolved
Hide resolved
5340840
to
9525a07
Compare
tests/spread/extensions/snaps/env-injector-hello/snap/snapcraft.yaml
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fixes.
I've just discovered another issue while testing.
d6ca9e4
to
c2bd475
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Thanks a lot.
Nevertheless, I do have concerns over the licensing of snaps using this extension. The extension adds a binary distribution to the snap which uses the following libraries:
- https://crates.io/crates/dotenv (MIT)
- https://docs.rs/http-body-util/latest/http_body_util/index.html (MIT)
- https://github.com/hyperium/hyper (MIT)
- https://github.com/hyperium/hyper-util (MIT)
- https://github.com/softprops/hyperlocal (MIT)
- https://github.com/serde-rs/serde (Apache 2.0 or MIT)
- https://docs.rs/serde_json/latest/serde_json/ (Apache 2.0 or MIT)
- https://github.com/tokio-rs/tokio (MIT)
We need to take a deeper look into this before flagging the extension as stable. I believe a snap using this extension needs to list MIT as one of its licenses and bundle a copy of the license files for all the above. A simpler solution would be to go back to the Bash implementation (at the expense of performance) with a royalty-free license such as CC0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, I think the extension and spread test look good, but I'm not sure how it will work without a configure hook.
Also, this will need a tests/unit/extensions/test_env_injector.py
.
I would like to return to one of the original criteria. We still need a snapcraft.io forum post about this idea to get input and feedback on this approach. We can adjust this PR so it gets published to a branch for testing.
We have done extensive research on the existing practices. The plan was to get this added as an experimental extension to allow widespread testing (assuming this is the whole point of experimental extensions) and validate the proposed configuration schema for the snap options keys. A forum post is a great idea to request community feedback. Will publishing to a branch make this extension available for testing via stable |
It would be available in a branch along the lines of update: we could also land this and #4926 into a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I think we need more discussion on this before landing to main, but even for a feature branch it would be interesting to refactor the test case to avoid splitting the extension test between the spread task scriptlet and verifications done inside C code, ideally both sides should reside in the task scriptlet (and maybe the C code can be entirely dropped).
Also compressing the rust binary can be pointless since the snap payload is already compressed (we should investigate other ways to reduce the binary size if a binary is to be used at all.)
Let's not forget that we still need this to be presented and discussed in the forum before landing it to main. In any case the injector code should reside in a separate repository instead of being part of the snapcraft codebase, and we must carefully consider if the extension is needed at all (the changes to snapcraft.yaml look too simple and can be easily added manually by the snap author). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From a code perspective, I think this is well written and tested. The documentation looks great too, although I didn't see anything about needing a configure hook.
Approving so we can land this into a feature branch this week to facilitate testing.
Update: I spoke with Claudio and he has some feedback before we move to a feature branch.
@mr-cal Good point! I've added a note section about the need for a configure hook. Thank you! |
@mr-cal Just saw your update! Thanks for letting me know! I'll be waiting for @cmatsuoka feedback. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! The remaining restriction I have is that the injector code should reside in a separate repository, as a stand-alone project, instead of being part of the Snapcraft codebase. That way, it can be maintained separately without requiring Snapcraft maintainers to own the injector code. After establishing it as a separate project we can land the extension code on a feature branch for tests.
Before landing on main we'll probably need to revisit the bash vs rust implementations of the injector to see if the former can be optimized to be faster, or the latter can be optimized to be smaller (e.g. using no_std and adding some local helpers).
@cmatsuoka The snappy-env (the exporter program) code is now separated from the extension and resides here: https://github.com/canonical/snappy-env |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've done some smoke testing and it looks good. There is a change to multipass command flags for spread testing which I think shouldn't be here.
Not a major issue, but env-exporter has been renamed to snappy-env (for good reasons since it also loads env files), but the surrounding refactoring is incomplete. It is still called env-exported in a few places. The extension adds a part named env-injector/env-injector
which could also change to env-injector/snappy-env
.
01c5c25
to
b2d8243
Compare
Add extension to inject environment variables into snaps by it's command-chain. This extension takes snap options and transform then into environment variables. For the env-exporter program, Rust Tier 2 toolchains are marked as unsupported. Signed-off-by: Lincoln Wallace <lincoln.wallace@canonical.com> * test: add spread tests for env-injector extension - add: spread test: * hello.c: application which consumes env variables * task.yaml: Tests to verify env-injector functionality Signed-off-by: Lincoln Wallace <lincoln.wallace@canonical.com>
Signed-off-by: Callahan Kovacs <callahan.kovacs@canonical.com>
- doc: Add line break on Pydoc - rust exporter: Fix error messages - spread: Add case to test app-specific envfile - spread: Fix test for verify alias overrite behavior - fix: typo error - fix: commentaries not being discarted in envfiles - feat: add logs to check test execution - fix: shellchek error - fix: use absolute path for envfiles - fix: address lint errors - fix: PEP8 formating - exporter program moved to: https://github.com/canonical/snappy-env Signed-off-by: Lincoln Wallace <lincoln.wallace@canonical.com> Co-authored-by: Farshid Tavakolizadeh <email@farshid.ws> Co-authored-by: Callahan <callahan.kovacs@canonical.com>
b2d8243
to
6fa6f67
Compare
777650b
to
6fa6f67
Compare
Signed-off-by: Lincoln Wallace <lincoln.wallace@canonical.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Now we can work on the exporter independently, and investigate if it could be no-std to reduce binary size, or experiment and compare to a fast shell script implementation.
tox run -m lint
?tox run -e test-py310
? (supported versions:py39
,py310
,py311
,py312
)Add extension to inject environment variables into snaps by its command-chain.
This extension takes snap options and transform them into environment variables