Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use GCS for Windows ccache #13183

Merged
merged 8 commits into from
Apr 20, 2023
Merged

Use GCS for Windows ccache #13183

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
32f009d
Use GCS for Windows ccache
GMNGeoffrey Apr 20, 2023
afe58a8
REVERT ME: Enable testing on my branch, skipping other jobs
GMNGeoffrey Apr 20, 2023
98d900f
Be consistent about quotes
GMNGeoffrey Apr 20, 2023
fda431a
Use correct secret name
GMNGeoffrey Apr 20, 2023
36109d4
Don't try to create credentials file
GMNGeoffrey Apr 20, 2023
9d34b72
Sure why give helpful errors when you can just silently do the wrong …
GMNGeoffrey Apr 20, 2023
76f539b
Use a ccache namespace
GMNGeoffrey Apr 20, 2023
227ade6
Revert "REVERT ME: Enable testing on my branch, skipping other jobs"
GMNGeoffrey Apr 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 12 additions & 33 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,19 +83,15 @@ jobs:
BUILD_DIR: build-windows
IREE_VULKAN_DISABLE: 1
steps:
- id: "gcp-auth"
name: "Authenticating to Google Cloud"
uses: "google-github-actions/auth@v1"
with:
token_format: "access_token"
credentials_json: "${{ secrets.IREE_OSS_GITHUB_RUNNER_BASIC_TRUST_SERVICE_ACCOUNT_KEY }}"
create_credentials_file: false
- name: "Checking out repository"
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
# Attempt to restore from cache unconditionally.
# Note: this will first try to grab a cache entry for this exact commit
# then it will fall back to the latest for any commit.
- name: "Fetching cache (CMake/ccache)"
uses: actions/cache/restore@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
with:
path: ${{ github.workspace }}/.ccache
key: ccache_all_windows_${{ github.sha }}
restore-keys: ccache_all_windows
# Fetch dependencies.
# TODO(scotttodd): Move some of these into a Docker image / add to PATH.
- name: "Updating git submodules"
run: git submodule update --init --jobs 8 --depth 1
- name: "Setting up Python"
Expand All @@ -114,30 +110,13 @@ jobs:
# Finally: build and run tests.
- name: "Building IREE"
env:
IREE_READ_REMOTE_CCACHE: 0
IREE_WRITE_REMOTE_CCACHE: 0
IREE_READ_LOCAL_CCACHE: 1
IREE_WRITE_LOCAL_CCACHE: ${{ needs.setup.outputs.write-caches }}
CCACHE_DIR: ${{ github.workspace }}/.ccache
# Cache size and compression level settings are a delicate balance.
# * A full build cache is around 2-5GB depending on compression level
# * Upload/download is slow (double compression may or may not help)
# * We have a limit of 10GB across all cached files per repository
# * Cache misses are quite costly:
# * 99% cache hits -> ~5 minutes to build
# * 20% cache hits -> ~15-20 minutes to build
CCACHE_MAXSIZE: 4G
CCACHE_COMPRESSLEVEL: 5
run: ./build_tools/cmake/build_all.sh "${BUILD_DIR}"
IREE_WRITE_REMOTE_CCACHE: ${{ needs.setup.outputs.write-caches }}
IREE_CCACHE_GCP_TOKEN: ${{ steps.gcp-auth.outputs.access_token }}
CCACHE_NAMESPACE: github-windows-2022-64core
run: |
./build_tools/cmake/build_all.sh "${BUILD_DIR}"
- name: "Testing IREE"
run: ./build_tools/cmake/ctest_all.sh "${BUILD_DIR}"
# Write cache (if configured to) after all other steps are finished.
- name: "Saving cache (CMake/ccache)"
if: needs.setup.outputs.write-caches == '1'
uses: actions/cache/save@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
with:
path: ${{ github.workspace }}/.ccache
key: ccache_all_windows_${{ github.sha }}

build_test_all_macos_arm64:
needs: setup
Expand Down