-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oci: fallback to fuse-overlayfs if kernel doesn't support unprivileged overlays and oci: support for writable extfs img overlay via fuse-overlayfs, from sylabs 1730 & 1740 #335
Commits on Jul 4, 2023
-
refactor: Launcher to interface & native.Launcher
In preparation for sylabs/singularity#1021, where we will add a placeholder OCI runtime launcher, refactor the existing code so that: * The internal/pkg/runtime/launcher package contains common option handling, utility functions, and a Launcher interface. * There is a launcher.native package containing the existing launch code for the native singularity runtime. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 193d25b - Browse repository at this point
Copy the full SHA 193d25bView commit details -
launcher: add placeholder OCI launcher
Accepts no options, fails if options provided. Implements an Exec method which does nothing. Fixes sylabs/singularity#1021 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for c4548b5 - Browse repository at this point
Copy the full SHA c4548b5View commit details -
actions: Add --oci flag to call OCI launcher
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for dffd2c8 - Browse repository at this point
Copy the full SHA dffd2c8View commit details -
launcher: fix OCI launcher supported option checks
Handle empty structs (not just nils). Handle always-set network, config file. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d62fe20 - Browse repository at this point
Copy the full SHA d62fe20View commit details -
oci: switch to runc for oci commands
Replace invocations of SingularityCE's OCI runtime engine with wrapped invocations of runc. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 99f68e8 - Browse repository at this point
Copy the full SHA 99f68e8View commit details -
oci: remove singularity oci_engine
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 00db0c7 - Browse repository at this point
Copy the full SHA 00db0c7View commit details -
oci: add conmon for detached flow
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for e604c9a - Browse repository at this point
Copy the full SHA e604c9aView commit details -
oci: use FindBin for conmon, runc
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for fed0ed9 - Browse repository at this point
Copy the full SHA fed0ed9View commit details -
oci: Update LICENSE docs and copyright lines
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 0cd88c2 - Browse repository at this point
Copy the full SHA 0cd88c2View commit details -
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3df5cf6 - Browse repository at this point
Copy the full SHA 3df5cf6View commit details -
oci: fix: ensure
oci exec
calls runc and e2e test checks thisAs noted in review, the `runc` call for `oci exec` was not actually being made. E2E test for `oci exec` was only checking return code, not for output so it passed. Fix these issues. In `e2e/suite.go` also move registry creation so that it is only performed for the man tests in PID+mount NS. It is not needed for the OCI/CGROUPS e2e tests and wastes CI time. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 5eb1db0 - Browse repository at this point
Copy the full SHA 5eb1db0View commit details -
oci: simplify AttachStreams type
As suggested in review the streams can be straight `io.Writer` / `io.Reader`. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 7952c83 - Browse repository at this point
Copy the full SHA 7952c83View commit details -
oci: chore: update missed copyright headers
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3d24eb1 - Browse repository at this point
Copy the full SHA 3d24eb1View commit details -
refactor: Move runc/conmon functionality out of app/singularity
In preparation for further OCI runtime work, move the code that calls out to runc/conmon from internal/app/singularity into internal/pkg/runtime/launcher/oci The oci.Launcher will make use of the basic OCI operations to run containers from its Exec function, so this is a good location while that work proceeds. The functions will be likely be modified considerably, and potentially moved again in future, as the design/implementation of the OCI runtime interaction is developed. The internal/app/singularity OCI* functions are left as a minimal shim layer, between the CLI layer and the launcher, at this time. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 588249e - Browse repository at this point
Copy the full SHA 588249eView commit details -
feat: run action for OCI bundle
As a first step toward run/shell/exec actions on native OCI images, implement a minimal `singularity run --oci mybundle` which: * Requires an on-disk bundle with appropriate `config.json`. * Runs this bundle using `crun` or `runc`. * Makes no attempt to handle any arguments or options. * Does not modify the `config.json` - i.e. it must match namespace / mapping requirements for rootless execution etc. At this stage, the functionality is essentially equivalent to `singularity oci run` and is not yet useful. The primary purpose of the PR is to refactor some of the code that passes args for launching a container. In addition, we now use `crun` in preference to `runc` if available. `crun` supports e.g. single uid->uid mapping in a usernamespace (without root mapping). Closes sylabs/singularity#598 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 955e9fb - Browse repository at this point
Copy the full SHA 955e9fbView commit details -
e2e: oci: add subtests, move help to parallel
Add subtest information to the large multi-step OCI test functions for easier debugging. Move the test of `oci help` out of the SEQ/ordered section. It can run in parallel, and only once regardless of cgroups managers available. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d5a958f - Browse repository at this point
Copy the full SHA d5a958fView commit details -
The Ubuntu packaged crun is very old. Use a release from GitHub instead. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for e6e9271 - Browse repository at this point
Copy the full SHA e6e9271View commit details -
feat: run OCI image sources via temporary bundle
When running containers in `--oci` mode, the argument to run is now an image reference corresponding to a native OCI format handled by containers/image, i.e. * docker:// * docker-archive: * docker-daemon: * oci-archive: * oci: The source image is extracted into a temporary OCI bundle, with a minimally valid configuration that: * Runs the process specified by CMD & ENTRYPOINT only. * Sets the environment specified by the image ENV only. The approach is very naive - we pull through Singularity's OCI blob cache into a temporary oci layout dir, before creating the bundle from it. Auth handling for registries is not yet wired up. There is duplication of various pieces of code from the build / SIF OCI flows as these are not easily exposed to the area we are working in. The intent of the PR, at this stage, is simply to allow e.g. singularity run --oci docker://sylabsio/lolcow Closes sylabs/singularity#1036 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 7daac21 - Browse repository at this point
Copy the full SHA 7daac21View commit details -
test: Add native OCI bundle verification test
Simple test to create a native oci bundle from the supported sources, and verify it is valid with runtime-tools/validate. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for b4a31ab - Browse repository at this point
Copy the full SHA b4a31abView commit details -
e2e: test for run --oci from oci sources
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 365100d - Browse repository at this point
Copy the full SHA 365100dView commit details -
chore: refactor native bundle to functional options
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 7bd5910 - Browse repository at this point
Copy the full SHA 7bd5910View commit details -
feat: exec / run args support for --oci mode
When using `run` or `exec` with the `--oci` runtime mode, accept arguments on the command line. For `run`, the arguments override any CMD specified by the image. For `exec`, the arguments replace ENTRYPOINT/CMD entirely, bypassing the process configuration in the image config. This mirrors the behavior of Singularity images today, via the exec and run runscripts - but is implemented in the OCI bundle config, rather than a script in the container. Closes sylabs/singularity#1024 Closes sylabs/singularity#1092 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f28acf7 - Browse repository at this point
Copy the full SHA f28acf7View commit details -
fix: return exit code from OCI launcher
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for e143519 - Browse repository at this point
Copy the full SHA e143519View commit details -
e2e: test --oci run/exec with args
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for fcacb62 - Browse repository at this point
Copy the full SHA fcacb62View commit details -
feat: enable shell for --oci mode
Enable `singularity shell --oci ...` with behavior matching native runtime, i.e. * Run shell set with SINGULARITY_SHELL or --shell * If not set, try /bin/bash --norc * If not available, use /bin/sh Closes sylabs/singularity#1025 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 24c9783 - Browse repository at this point
Copy the full SHA 24c9783View commit details -
fix: correct uid/gid non-root mapping
Ensure e2e tests for oci actions use user profile. Set uid/gid mappings explicitly. We need to do this anyone, going forward, but here it works around: containers/crun#1072 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 92f2ddb - Browse repository at this point
Copy the full SHA 92f2ddbView commit details -
chore: Update deps for CI, packages, docs
runc -> crun Need uidmap on Debian Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f5e51c3 - Browse repository at this point
Copy the full SHA f5e51c3View commit details -
e2e: use root and user OCI profiles in action tests
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for af57b65 - Browse repository at this point
Copy the full SHA af57b65View commit details -
pkg: Update / tidy rpm spec file
Tidy up the rpm spec file. * Remove the manual handling of build root, GOPATH etc. Not needed as we are using go modules now. * Remove redundant explicit deps. * Fix crun -> runc dep for EL7. * Use rpm make_xxx macros instead of direct make calls. * Ensure all directories created are owned by package. Fixes sylabs/singularity#1142 Fixes sylabs/singularity#1141 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 4563242 - Browse repository at this point
Copy the full SHA 4563242View commit details -
Update INSTALL.md - fix typo from PR review.
Co-authored-by: Mike Frisch <mikef17@gmail.com> Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for b8d648d - Browse repository at this point
Copy the full SHA b8d648dView commit details -
pkg: fix Ubuntu 18.04 deb build
If `crun` is not available, require `runc`. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3e3e6fd - Browse repository at this point
Copy the full SHA 3e3e6fdView commit details -
oci: move some code to launcher, use explicit config
Move ID mapping and process setup code to the launcher, out of the oci bundle package. Add a minimal config for `--oci` mode, rather than starting with the default OCI config. Add explicit configuration of mounts, with `tmpfs` mount locations matching the native runtime with `--containall / --compat`. Ensure ID mapping is always explicit (including for root). Add tests touching the tmpfs locations. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f4c677c - Browse repository at this point
Copy the full SHA f4c677cView commit details -
oci: Add --fakeroot support to --oci mode
Initial --fakeroot support for --oci mode. Mirrors behavior with --compat / --contain. Closes sylabs/singularity#1035 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 2cdf179 - Browse repository at this point
Copy the full SHA 2cdf179View commit details -
fix: use conf file value for oci tmpfs size
Use the configuration file sessiondir max size value for --oci mode tmpfs mounts. Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems. Fixes sylabs/singularity#1140 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 5f196e9 - Browse repository at this point
Copy the full SHA 5f196e9View commit details -
feat: oci: support namespace flags
Support namespace request CLI options. * --ipc - no effect, always used in --oci mode. * --net - only supported with --network none. * --pid - no effect, always used in --oci mode. * -u / --userns - only effective for root, non-root always uses user ns. * --uts Add info logging where the option is redundant. Closes sylabs/singularity#1026 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 8d8de03 - Browse repository at this point
Copy the full SHA 8d8de03View commit details -
chore: refactor for easier testing
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for de885a3 - Browse repository at this point
Copy the full SHA de885a3View commit details -
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 14405c8 - Browse repository at this point
Copy the full SHA 14405c8View commit details -
e2e: minimal --oci namespace request tests
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 53ac7f8 - Browse repository at this point
Copy the full SHA 53ac7f8View commit details -
fix: Don't set Process.Terminal in oci launcher if no term
If we are running such that stdin is not a terminal, then the OCI runtime config should have Process.Terminal=false to avoid errors. https://github.com/opencontainers/runc/blob/main/docs/terminals.md#issues Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f73e208 - Browse repository at this point
Copy the full SHA f73e208View commit details -
chore: move BindPath code out of runtime package
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 930836b - Browse repository at this point
Copy the full SHA 930836bView commit details -
feat: oci: enable bind mounts via --bind, --mount
Implement support for bind mounts (rw & ro) specified using -B/--bind and --mount on the singularity command line. Fixes sylabs/singularity#1027 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f2a4cf2 - Browse repository at this point
Copy the full SHA f2a4cf2View commit details -
e2e: port relevant --bind/mount action tests for --oci
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 4052b17 - Browse repository at this point
Copy the full SHA 4052b17View commit details -
fix: oci: disallow relative bind destinations
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for a63047a - Browse repository at this point
Copy the full SHA a63047aView commit details -
chore: fix stutter bind.Bindxxxx lint
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3578fe6 - Browse repository at this point
Copy the full SHA 3578fe6View commit details -
feat: oci: support --env option in --oci mode
* Merge image config ENV and env vars requested by user with the --env CLI option. * Set default SINGULARITY_CONTAINER and SINGULARITY_NAME env variables. * Set default LD_LIBRARY_PATH to be used later for library injection (this is a singularity default). Fixes sylabs/singularity#1029 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 8cf8779 - Browse repository at this point
Copy the full SHA 8cf8779View commit details -
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for ed191d1 - Browse repository at this point
Copy the full SHA ed191d1View commit details -
feat: oci: add SINGULARITYENV_ handling for --oci mode
Pass SINGULARITYENV_ prefixed environment variables into container in --oci mode. Fixes sylabs/singularity#1031 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 75ec7b4 - Browse repository at this point
Copy the full SHA 75ec7b4View commit details -
e2e: add SINGULARITYENV_ tests for --oci mode
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 1da3b41 - Browse repository at this point
Copy the full SHA 1da3b41View commit details -
feat: oci: enable --env-file in --oci mode
Allow --env-file to be used to provide environment variables in a file, when running a container in --oci mode. We use the same approach as the native runtime for compatibility. The env file is evaluated in the embedded shell interpreter, but starting with an empty environment. This handles quoting, comments etc. for us, and keeps maximum compatibility with the existing handling. Fixes sylabs/singularity#1030 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 1c527b4 - Browse repository at this point
Copy the full SHA 1c527b4View commit details -
e2e: add --env-file tests for --oci mode
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 31ea98e - Browse repository at this point
Copy the full SHA 31ea98eView commit details -
doc: Add CHANGELOG entry for
--oci
Now that sufficient functionality is in place for `--oci` mode, add a CHANGELOG entry for the current state. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d896ab6 - Browse repository at this point
Copy the full SHA d896ab6View commit details -
fix: oci: ensure user/group entries in container
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3ec4133 - Browse repository at this point
Copy the full SHA 3ec4133View commit details -
feat: oci: Support --rocm with --oci mode
Under --oci mode, allow --rocm, which will bind ROCm devices, libraries, and binaries into the container. Fixes sylabs/singularity#1034 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 5f601bf - Browse repository at this point
Copy the full SHA 5f601bfView commit details -
e2e: minimal --rocm --oci test
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 1c653b9 - Browse repository at this point
Copy the full SHA 1c653b9View commit details -
fix: rocm: update rocmliblist and fix e2e tests
`rocminfo` now needs `lsmod` and libdrm libdrm_amdgpu. Bind the former in tests, the libraries from rocmliblist.conf. We can now use Ubuntu 22.04 for ROCm tests. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 33b061b - Browse repository at this point
Copy the full SHA 33b061bView commit details -
doc: --rocm --oci CHANGELOG entry
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 8313a4d - Browse repository at this point
Copy the full SHA 8313a4dView commit details -
oci: support legacy --nv bind mode
Closes sylabs/singularity#1033 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 17fcf13 - Browse repository at this point
Copy the full SHA 17fcf13View commit details -
chore: refactor runtime spec handling out of bundle
Bring all handling of the runtime spec out of the native oci bundle package, up into the oci launcher. We obtain the bundle's image spec after it is downloaded / extracted. The launcher then computes the correct Process config and updates the bundle with it. This is required so that we can handle an image USER in the launcher. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for cdcf38e - Browse repository at this point
Copy the full SHA cdcf38eView commit details -
oci: inspection of image user in bundle
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 92beb96 - Browse repository at this point
Copy the full SHA 92beb96View commit details -
feat: oci: honor USER in image config
When a USER is specified in the image config: * If running unprivileged, ensure the inner uid / gid mapping results in the container process running as the USER, by default. * If running privileged, run as the USER, by default. Fixes sylabs/singularity#77 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 073da5d - Browse repository at this point
Copy the full SHA 073da5dView commit details -
fix: lack of passwd / group should be warning in --oci mode
In the --oci launcher, When a minimal container doesn't have an `/etc/passwd` or `/etc/groups` file then don't fail with a fatal error. Instead warn like the native runtime does. Fixes sylabs/singularity#1286 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 4a03673 - Browse repository at this point
Copy the full SHA 4a03673View commit details -
oci: pass in SystemContext for image handling
Honour the DOCKER_HOST, `--no-https`, and auth configuration. Fixes sylabs/singularity#1220 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for a489ef6 - Browse repository at this point
Copy the full SHA a489ef6View commit details -
oci: Allow disabled cache in oci launcher
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 0137ee3 - Browse repository at this point
Copy the full SHA 0137ee3View commit details -
fix: e2e: docker rmi ref, not uri
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 1b3230f - Browse repository at this point
Copy the full SHA 1b3230fView commit details -
fix: change option for oci launcher unsupported option test
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for c7ec495 - Browse repository at this point
Copy the full SHA c7ec495View commit details -
feat: oci: support cgroups requests in --oci mode
Honor the --apply-cgroups and individual cgroups resources flags when running in OCI mode. The launcher instructs runc/crun to create a named cgroup with specified LinuxResources in the config.json. runc/crun must be called with the `--systemd-cgroup` flag when using systemd as cgroup manager. Closes sylabs/singularity#1032 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for fb2ec7d - Browse repository at this point
Copy the full SHA fb2ec7dView commit details -
fix: fall back to cgroupfs for OCI non-root, non cgroups v2
For the `OCI` command group, we need cgroups v2 unified mode in order to call `runc` with `--systemd-cgroup` as a non-root user. Fall back to cgroupfs operation if this constraint is not satisified. This fixes a failure when running a container using `runc`. `crun` was falling back to cgroupfs itself, even when `--systemd-cgroup` was specified. Fixes sylabs/singularity#1408 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 01c9820 - Browse repository at this point
Copy the full SHA 01c9820View commit details -
oci: drop warning about runc functionality
Drop the warning that not all OCI functionality is supported with runc. Some things turned out to be SingularityCE bugs. Some things are very version specific. A new enough runc will work. Fixes sylabs/singularity#1425 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d59ca94 - Browse repository at this point
Copy the full SHA d59ca94View commit details -
rpm: fix: correct BuildRequires/Requires for SLES
* On SLES we intend to require runc, not crun. * squashfs had a duplicate Requires, instead of a BuildRequires. Fixes sylabs/singularity#1453 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 267615e - Browse repository at this point
Copy the full SHA 267615eView commit details -
e2e: refactor test image handling
Refactor test image handling so that: * We create the OCI archive, and Docker archive, used in the e2e tests with a container/image Copy from our local registry busybox image. * We use DockerHub credentials for e2e.CopyOCIImage. * We replace a couple of uses of `library://` images with our local registry ORAS SIF, avoiding some additional network access. Fixes sylabs/singularity#1364 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3103743 - Browse repository at this point
Copy the full SHA 3103743View commit details -
oci: implemented CDI device mapping (sylabs/singularity#1459)
* oci: implemented CDI device mapping * first batch of post-comments revisions * revisions following elezar's comments * added `--device` to CHANGELOG * fixed out-of-order calls to addCDIDevices & Update * added sync.Once getting & refreshing CDI registry * cdi unit-test stub * added some more cdi unit-tests * more unit-tests * better testing for empty mounts lists * finishing touches on cdi unit-test * first stab at cdi e2e-test * DT's temp fix for userns mapping limitation * ignore VSCode debugging build targets * renamed CDI json template file * added initial deviceNode testing * changed flag to --cdi-dirs (from --cdidirs) Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 7117ded - Browse repository at this point
Copy the full SHA 7117dedView commit details -
oci: support --hostname (sylabs/singularity#1495)
* oci: support --hostname * generalize --hostname inferring --uts across native & oci Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 2b2572c - Browse repository at this point
Copy the full SHA 2b2572cView commit details -
oci: support --scratch (sylabs/singularity#1498)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 2b03310 - Browse repository at this point
Copy the full SHA 2b03310View commit details -
oci: support --pwd (sylabs/singularity#1558)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for c3f315d - Browse repository at this point
Copy the full SHA c3f315dView commit details -
oci: support --home (sylabs/singularity#1500)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f6c60a8 - Browse repository at this point
Copy the full SHA f6c60a8View commit details -
oci: improvements to cdi support
* added --cdi-dirs to changelog * improved comments in actionOciCdi e2e-test Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 0e2dba6 - Browse repository at this point
Copy the full SHA 0e2dba6View commit details -
fix: oci: explicitly request userns for inner reverse idmap
When running in --oci mode as a normal user, Singularity sets up an outer userns that provides an unprivileged root. When --fakeroot is not also requested, runc / crun then map back to the user's own uid/gid via an inner reverse mapping specified in config.json. This mapping was not being applied correctly, as a userns was not requested in the config.json for all situations where a reverse mapping was requested. Ensure that if we specify a mapping in config.json then we request a userns so that it is applied. Cover this behavior in an e2e-test by writing to a host dir from the container, and checking ownership of the file on the host. Fixes sylabs/singularity#1517 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for de564cb - Browse repository at this point
Copy the full SHA de564cbView commit details -
fix: oci: reverse uid/gid maps now honour target IDs
In `getReverseUserMaps`,we were taking `targetUID` and `targetGID` parameters that define the UID and GID the container will be entereed as. However, the mappings returned were based on the current host UID and GID, and not the target IDs. Ensure that the user maps use the `targetUID` and `targetGID`. Split the function up so that the core computation of the user maps can be tested. The tests help to explain what the intended functionality here is... which is beneficial as it's somewhat complex. Fixes sylabs/singularity#1519 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for ff5bd89 - Browse repository at this point
Copy the full SHA ff5bd89View commit details -
fix: oci: enter cgroup before executing crun as non-root
When executed from a root-owned cgroup, such as the session scope resulting from a bare ssh login, crun will fail to create our requested container cgroup. If we are running as non-root, create and move into a user-owned cgroup, so that there's a common user-owned ancestor. This avoids the `crun` error. Note that no workaround is needed for `runc` as it is able to create the requested container cgroup without any issue. Fixes sylabs/singularity#1538 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 569b626 - Browse repository at this point
Copy the full SHA 569b626View commit details -
fix: TERM not set in OCI containers
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3ee67e0 - Browse repository at this point
Copy the full SHA 3ee67e0View commit details -
oci: support resolv.conf and --dns
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 27de69c - Browse repository at this point
Copy the full SHA 27de69cView commit details -
warn instead of failing when container lacks /etc (cf. issue1286)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 9803987 - Browse repository at this point
Copy the full SHA 9803987View commit details -
added rudimentary tests for --dns
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 359200e - Browse repository at this point
Copy the full SHA 359200eView commit details -
fix: oci: honour config passwd / config group directives
Ensure that the `config passwd` and `config group` directives in `singularity.conf` are honoured in `--oci` mode. Add e2e config tests for OCI mode that test this. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 6cc1809 - Browse repository at this point
Copy the full SHA 6cc1809View commit details -
fix: oci: honour mount proc/sys/tmp/home directives
When mount proc/sys/tmp/home directives are set to no in singularity.conf, ensure we don't mount at those locations. Also enables a user bind control test, for which the implementation is already in place and working. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3527ad6 - Browse repository at this point
Copy the full SHA 3527ad6View commit details -
fix: oci: honour config resolv_conf directive
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d4220cd - Browse repository at this point
Copy the full SHA d4220cdView commit details -
fixed typo (resolve.conf -> resolv.conf) (sylabs/singularity#1566)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for fc9ad81 - Browse repository at this point
Copy the full SHA fc9ad81View commit details -
pull HashingListSubtract into (new) high-level util package (sylabs/s…
…ingularity#1562) * pull HashingListSubtract into (new) high-level util package * to-be-squashed: addressing DT's comments Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for b71fc58 - Browse repository at this point
Copy the full SHA b71fc58View commit details -
fix: oci: Don't create cgroup for crun on v1 / cgroupfs
If we are running under cgroups v1 or with the cgroupfs manager (i.e. not systemd as cgroup manager), do not attempt to enter a cgroup at startup with crun. We cannot create a cgroup unprivileged in this situation. Under cgroups v1, crun will not perform the cgroups manipulation that leads to the issue we worked around in apptainer#1539. Any other issue with the cgroup that we are in at launch cannot be rectified, either. Fixes sylabs/singularity#1569 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for ad089bc - Browse repository at this point
Copy the full SHA ad089bcView commit details -
fix: Remove files with restrictive perms from --oci temp rootfs
When we run a container in `--oci` mode, we have a temporary rootfs that must be removed when the container exits. A container can contain files / dirs with restrictive permissions that prevent direct removal. Use fs.ForceRemoveAll, which will chmod & remove anything that os.RemoveAll is unable to remove. Fixes sylabs/singularity#1586 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 2decb88 - Browse repository at this point
Copy the full SHA 2decb88View commit details -
oci: Enable --writable-tmpfs behaviour by default
The `--oci` mode intends to follow behaviour that the native runtime implements when run with `--compat`. One missing aspect is that `--compat` sets `--writable-tmpfs`, where the container rootfs is made writable with a tmpfs backed overlay. This PR: - Introduces a simple wrapping of the `oci run` sub-command as `oci run-wrapped`. This hidden command implements prep / cleanup steps that must take place in a userns for non-root `--oci` execution. - Switches the oci launcher to calling `oci run-wrapped` instead of `oci-run`. - Adds a tmpfs based overlay creation function for OCI bundles. - Includes the tmpfs overlay creation in the `oci run-wrapped` flow. - Copies the native runtime `--compat` e2e tests to OCI mode. Fixes sylabs/singularity#1621 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for d78657a - Browse repository at this point
Copy the full SHA d78657aView commit details -
1481 support pwd in oci mode (sylabs/singularity#1496)
* oci: support --pwd * replaced --pwd with --cwd (leaving former as synonym), adjusted code accordingly throughout * to-be-squashed: addressing DT's Mar 30 comments Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for aa691e5 - Browse repository at this point
Copy the full SHA aa691e5View commit details -
oci: fix --cwd/--pwd and actionOciExec() e2e-test
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 712ed4c - Browse repository at this point
Copy the full SHA 712ed4cView commit details -
oci: fix --home when running as root or fakeroot (sylabs/singularity#…
…1530) Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 58be7e5 - Browse repository at this point
Copy the full SHA 58be7e5View commit details -
oci: fix to not override HOME when container specifies USER (+ e2e te…
…st of this) Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f033d46 - Browse repository at this point
Copy the full SHA f033d46View commit details -
oci: support --overlay (sylabs/singularity#1659)
* oci: support --overlay * support for multiple overlays, other revisions * fixup: deduplicate RunWrapped logic * lots of refactoring and cleanup * remove leftover debug-related panic call * cleanup comments, fix small issues w/erroring --------- Co-authored-by: David Trudgian <david.trudgian@sylabs.io> Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 05a9efc - Browse repository at this point
Copy the full SHA 05a9efcView commit details -
wrap in writabletmpfs overlay if there's only read-only overlay(s)
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 061b653 - Browse repository at this point
Copy the full SHA 061b653View commit details -
improved unmounting of user-supplied overlays
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for f96e620 - Browse repository at this point
Copy the full SHA f96e620View commit details -
added e2e testing for 1670 regression
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 002535d - Browse repository at this point
Copy the full SHA 002535dView commit details -
oci: add e2e OCIRoot test to check proper unmounting of overlays
(sylabs/singularity#1678) Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 3bcbf68 - Browse repository at this point
Copy the full SHA 3bcbf68View commit details -
chg --workdir usage msg: remove mention of $HOME
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for c8508ef - Browse repository at this point
Copy the full SHA c8508efView commit details -
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 11992df - Browse repository at this point
Copy the full SHA 11992dfView commit details -
rel. path --workdir/--scratch fix; code style improvements
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 166b35b - Browse repository at this point
Copy the full SHA 166b35bView commit details -
fix: correct Stdin wire-up in runc/crun execs
When runc/crun is called from oci_runc_linux.go, Stdin was incorrectly connected for the various runc/crun operations. * Non-interactive operations such as resume / kill don't need Stdin. * Interactive operations (run/exec) had cmd.Stdin incorrectly set to os.Stdout. This prevented OCI containers from receiving input from pipes, redirection, etc. Fixes sylabs/singularity#1712 Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 56364e4 - Browse repository at this point
Copy the full SHA 56364e4View commit details -
e2e: port action STDPIPE tests to --oci mode
These were missed, resulting in sylabs/singularity#1712 not being caught by the e2e suite. Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 7e357b0 - Browse repository at this point
Copy the full SHA 7e357b0View commit details -
oci: support --overlay of bare images (sylabs/singularity#1699)
OCI-mode: support mounting of squashfs images (read-only) and extfs images (read-write or read-only) with --overlay, alongside the already existing functionality of directory-based overlays. Squashed merge of: * prepare data structures for non-dir overlays * fix e2e test (no more "auto-create overlay dir" functionality) * switch to using image.Init() to analyze image files * added caching mechanism to FindBin * first working version with squashfs support * refactor funcs into methods of OverlayItem where appropriate * standardized naming + added lots of comments * addressing first round of review comments * refactor: move non-OCI-dep. code to internal/pkg/util/fs/overlay * initial support for extfs overlay (readonly) * resurrect testing of write to persistent overlay * fix bug in writable overlay, introduced in course of refactor * removed caching mechanism from FindBin * addressing second round of review comments, except tests * move calling of prepareWritableOverlay() into Item.Mount() * removed redundant error-string wrapping * overlay.Item unit-test * overlay.Set unit-test * e2e tests for oci image overlays, misc. testing improvements Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 8691b8e - Browse repository at this point
Copy the full SHA 8691b8eView commit details -
move overlay test images to test/ subtree
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 16972cb - Browse repository at this point
Copy the full SHA 16972cbView commit details -
Disable check_pkg_no_buildcfg temporarily
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for ccdb465 - Browse repository at this point
Copy the full SHA ccdb465View commit details -
fix rel. path --workdir with --scratch, add native and oci e2e tests
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 765cc92 - Browse repository at this point
Copy the full SHA 765cc92View commit details -
error out when workdir can't be converted to abs path
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 37e24e5 - Browse repository at this point
Copy the full SHA 37e24e5View commit details -
privileged cleanup in (oci)RelWorkdirScratch tests
Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 1516258 - Browse repository at this point
Copy the full SHA 1516258View commit details
Commits on Jul 6, 2023
-
Merge pull request #334 from edytuk/sylabs1694v3
fix rel. path --workdir with --scratch, add oci e2e tests, from sylabs 1694
Configuration menu - View commit details
-
Copy full SHA for d115f91 - Browse repository at this point
Copy the full SHA d115f91View commit details -
oci: fallback to fuse-overlayfs if kernel doesn't support unprivilege…
…d overlays (sylabs/singularity#1730) * fallback to fuse-overlayfs if kernel doesn't support unpriv overlays * addressed first round of review comments * fix code-flow in UnprivOverlaysSupported() for unsupported case Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for 12fb599 - Browse repository at this point
Copy the full SHA 12fb599View commit details
Commits on Jul 7, 2023
-
oci: support for writable extfs img overlay via fuse-overlayfs
(sylabs/singularity#1740) * oci: support for writable extfs img overlay via fuse-overlayfs * added unit- and e2e-tests * fixed exposure of "upper" & "work" subdir in readonly overlays * added fix for file-ownership in FUSE-mounted images + e2e test of fix * fix handling of "upper" in :ro overlay dirs, adjust e2e tests Signed-off-by: Edita Kizinevic <edita.kizinevic@cern.ch>
Configuration menu - View commit details
-
Copy full SHA for c3fa965 - Browse repository at this point
Copy the full SHA c3fa965View commit details -
do not re-use the rootfs name for overlay upper dir
Signed-off-by: Dave Dykstra <2129743+DrDaveD@users.noreply.github.com>
Configuration menu - View commit details
-
Copy full SHA for d4fdf44 - Browse repository at this point
Copy the full SHA d4fdf44View commit details