Skip to content

Commit

Permalink
Auto merge of rust-lang#123678 - DianQK:docker-local, r=Kobzol
Browse files Browse the repository at this point in the history
Improve the experience of running Docker locally

When running locally, the absence of the `GITHUB_STEP_SUMMARY` environment variable will lead to the following error:

```
::endgroup::
./src/ci/docker/run.sh: line 349: : No such file or directory
```

I've also changed the output artifacts directory to `obj/$image_name`, allowing me to easily run all images locally. We always encounter various strange issues when modifying the test cases in the `codegen` directory.

r? Kobzol cc `@saethlin`
  • Loading branch information
bors committed Apr 17, 2024
2 parents b758149 + 86e939e commit 6c6b302
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 14 deletions.
11 changes: 4 additions & 7 deletions src/ci/docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,17 @@ for example:
./src/ci/docker/run.sh x86_64-gnu
```

Images will output artifacts in an `obj` dir at the root of a repository. Note
Images will output artifacts in an `obj/$image_name` dir at the root of a repository. Note
that the script will overwrite the contents of this directory.

To match conditions in rusts CI, also set the environment variable `DEPLOY=1`, e.g.:
```
DEPLOY=1 ./src/ci/docker/run.sh x86_64-gnu
```

**NOTE**: Re-using the same `obj` dir with different docker images with
the same target triple (e.g. `dist-x86_64-linux` and `dist-various-1`)
may result in strange linker errors, due shared library versions differing between platforms.

If you encounter any issues when using multiple Docker images, try deleting your `obj` directory
before running your command.
**NOTE**: In CI, the script outputs the artifacts to the `obj` directory,
while locally, to the `obj/$image_name` directory. This is primarily to prevent
strange linker errors when using multiple Docker images.

## Filesystem layout

Expand Down
20 changes: 13 additions & 7 deletions src/ci/docker/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,24 @@ ci_dir="`dirname $script_dir`"
src_dir="`dirname $ci_dir`"
root_dir="`dirname $src_dir`"

objdir=$root_dir/obj
source "$ci_dir/shared.sh"

if isCI; then
objdir=$root_dir/obj
else
objdir=$root_dir/obj/$image
fi
dist=$objdir/build/dist


if [ -d "$root_dir/.git" ]; then
IS_GIT_SOURCE=1
fi

source "$ci_dir/shared.sh"

CACHE_DOMAIN="${CACHE_DOMAIN:-ci-caches.rust-lang.org}"

if [ -f "$docker_dir/$image/Dockerfile" ]; then
if [ "$CI" != "" ]; then
if isCI; then
hash_key=/tmp/.docker-hash-key.txt
rm -f "${hash_key}"
echo $image >> $hash_key
Expand Down Expand Up @@ -102,7 +106,7 @@ if [ -f "$docker_dir/$image/Dockerfile" ]; then
CACHE_IMAGE_TAG=${REGISTRY}/${REGISTRY_USERNAME}/rust-ci-cache:${cksum}

# On non-CI jobs, we don't do any caching.
if [[ "$CI" == "" ]];
if ! isCI;
then
retry docker build --rm -t rust-ci -f "$dockerfile" "$context"
# On PR CI jobs, we don't have permissions to write to the registry cache,
Expand Down Expand Up @@ -289,7 +293,7 @@ else
command=(/checkout/src/ci/run.sh)
fi

if [ "$CI" != "" ]; then
if isCI; then
# Get some needed information for $BASE_COMMIT
#
# This command gets the last merge commit which we'll use as base to list
Expand Down Expand Up @@ -339,7 +343,9 @@ docker \
rust-ci \
"${command[@]}"

cat $objdir/${SUMMARY_FILE} >> "${GITHUB_STEP_SUMMARY}"
if isCI; then
cat $objdir/${SUMMARY_FILE} >> "${GITHUB_STEP_SUMMARY}"
fi

if [ -f /.dockerenv ]; then
rm -rf $objdir
Expand Down

0 comments on commit 6c6b302

Please sign in to comment.