Skip to content

Commit

Permalink
.github: Add options to force unzip artifacts (pytorch#56929)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: pytorch#56929

Artifacts were failing to unzip since they already existed in the
current tree so this just forces the zip to go through no matter what

Was observing that test phases will fail if attempting to zip over an already existing directory, https://github.com/pytorch/pytorch/runs/2424525136?check_suite_focus=true

In the long run however it'd be good to have these binaries built out as part of the regular cmake process instead of being one off builds like they are now

**NOTE**: This wouldn't be an issue if `--ephemeral` workers was a thing, see: actions/runner#660

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Test Plan: Imported from OSS

Reviewed By: janeyx99

Differential Revision: D28004271

Pulled By: seemethere

fbshipit-source-id: c138bc85caac5d411a0126d27cc42c60fe88de60
  • Loading branch information
seemethere authored and Kushashwa Shrimali committed May 18, 2021
1 parent acbb5cd commit 0bc9c77
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 6 deletions.
9 changes: 7 additions & 2 deletions .github/templates/linux_ci_workflow.yml.in
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ jobs:
- name: Chown workspace
run: |
# Ensure the working directory gets chowned back to the current user
docker run --rm -v "$(pwd)":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
docker run --rm -v "$(pwd)/../":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
- name: Checkout PyTorch
uses: actions/checkout@v2
- name: Log in to ECR
Expand Down Expand Up @@ -150,13 +150,18 @@ jobs:
;;
esac
echo "SHM_SIZE=${shm_size}" >> "${GITHUB_ENV}"
- name: Remove test binary build directories
run: |
rm -rf ../custom-op-build
rm -rf ../custom-backend-build
rm -rf ../jit-hook-build
- uses: actions/download-artifact@v2
name: Download PyTorch Build Artifacts
with:
name: ${{ env.BUILD_ENVIRONMENT }}
- name: Unzip artifacts
run: |
(cd "${GITHUB_WORKSPACE}/../" && unzip -q pytorch/artifacts.zip)
(cd "${GITHUB_WORKSPACE}/../" && unzip -qof pytorch/artifacts.zip)
- name: Output disk space left
run: |
sudo df -H
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ jobs:
- name: Chown workspace
run: |
# Ensure the working directory gets chowned back to the current user
docker run --rm -v "$(pwd)":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
docker run --rm -v "$(pwd)/../":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
- name: Checkout PyTorch
uses: actions/checkout@v2
- name: Log in to ECR
Expand Down Expand Up @@ -150,13 +150,18 @@ jobs:
;;
esac
echo "SHM_SIZE=${shm_size}" >> "${GITHUB_ENV}"
- name: Remove test binary build directories
run: |
rm -rf ../custom-op-build
rm -rf ../custom-backend-build
rm -rf ../jit-hook-build
- uses: actions/download-artifact@v2
name: Download PyTorch Build Artifacts
with:
name: ${{ env.BUILD_ENVIRONMENT }}
- name: Unzip artifacts
run: |
(cd "${GITHUB_WORKSPACE}/../" && unzip -q pytorch/artifacts.zip)
(cd "${GITHUB_WORKSPACE}/../" && unzip -qof pytorch/artifacts.zip)
- name: Output disk space left
run: |
sudo df -H
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/pytorch-linux-xenial-py3.6-gcc5.4.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ jobs:
- name: Chown workspace
run: |
# Ensure the working directory gets chowned back to the current user
docker run --rm -v "$(pwd)":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
docker run --rm -v "$(pwd)/../":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
- name: Checkout PyTorch
uses: actions/checkout@v2
- name: Log in to ECR
Expand Down Expand Up @@ -150,13 +150,18 @@ jobs:
;;
esac
echo "SHM_SIZE=${shm_size}" >> "${GITHUB_ENV}"
- name: Remove test binary build directories
run: |
rm -rf ../custom-op-build
rm -rf ../custom-backend-build
rm -rf ../jit-hook-build
- uses: actions/download-artifact@v2
name: Download PyTorch Build Artifacts
with:
name: ${{ env.BUILD_ENVIRONMENT }}
- name: Unzip artifacts
run: |
(cd "${GITHUB_WORKSPACE}/../" && unzip -q pytorch/artifacts.zip)
(cd "${GITHUB_WORKSPACE}/../" && unzip -qof pytorch/artifacts.zip)
- name: Output disk space left
run: |
sudo df -H
Expand Down

0 comments on commit 0bc9c77

Please sign in to comment.