Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand Linux test matrix #4454

Merged
merged 24 commits into from
Apr 24, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2287430
Factor out composite action for build step
thejohnfreeman Dec 22, 2022
39a925d
Python is no longer used directly
thejohnfreeman Dec 22, 2022
13f98a5
Pass name of profile
thejohnfreeman Dec 22, 2022
1cd1190
Fix build script
thejohnfreeman Dec 22, 2022
bc214c6
Target self-hosted runner
thejohnfreeman Dec 22, 2022
5a970e3
Expand matrix on compiler, configuration, and unity
thejohnfreeman Dec 22, 2022
f6afe68
Set path to compiler in profile
thejohnfreeman Dec 22, 2022
9c9649a
Add missing profile name
thejohnfreeman Dec 22, 2022
02ca3ef
Move matrix objects to include section
thejohnfreeman Dec 22, 2022
12f0ecc
Use [buildenv] instead of [env]
thejohnfreeman Dec 22, 2022
35b6460
Separate jobs to build Conan dependency caches
thejohnfreeman Dec 23, 2022
1ca0b11
Cannot set [buildenv] from command line
thejohnfreeman Dec 24, 2022
6255be8
Shell-quote JSON on command line
thejohnfreeman Dec 24, 2022
6119069
Add missing profile name
thejohnfreeman Dec 24, 2022
149f484
Substitute for missing environment variable
thejohnfreeman Dec 24, 2022
49e0135
Archive and compress cache artifact
thejohnfreeman Dec 24, 2022
55b14e7
Fix artifact download path
thejohnfreeman Dec 25, 2022
95f360c
Try heavy runners
thejohnfreeman Jan 12, 2023
ff47ee2
Try again
thejohnfreeman Jan 12, 2023
77a5512
Merge branch develop
thejohnfreeman Feb 3, 2023
53e6bb8
Merge upstream develop
thejohnfreeman Mar 8, 2023
36e104a
Fix Conan export in build action
thejohnfreeman Mar 9, 2023
bfbcaff
Try to restore macos workflow
thejohnfreeman Mar 16, 2023
f9b9cfa
Migrate from deprecated set-output command
thejohnfreeman Mar 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/actions/build/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: build
inputs:
generator:
default: null
configuration:
required: true
cmake-args:
default: null
# An implicit input is the environment variable `build_dir`.
runs:
using: composite
steps:
- name: export custom recipes
shell: bash
run: conan export external/snappy snappy/1.1.9@
- name: install dependencies
shell: bash
run: |
mkdir ${build_dir}
cd ${build_dir}
conan install \
--output-folder . \
--build missing \
--settings build_type=${{ inputs.configuration }} \
..
- name: configure
shell: bash
run: |
cd ${build_dir}
cmake \
${{ inputs.generator && format('-G {0}', inputs.generator) || '' }} \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${{ inputs.configuration }} \
${{ inputs.cmake-args }} \
..
- name: build
shell: bash
run: |
cmake \
--build ${build_dir} \
--config ${{ inputs.configuration }} \
--parallel ${NUM_PROCESSORS:-$(nproc)}
148 changes: 83 additions & 65 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,94 +3,112 @@ on: [push, pull_request]

jobs:

test:
dependencies:
strategy:
fail-fast: false
matrix:
platform:
- ubuntu-latest
- macos-12
generator:
- Ninja
- linux
compiler:
- gcc
- clang
configuration:
- Debug
- Release
runs-on: ${{ matrix.platform }}
env:
build_dir: .build
include:
- compiler: gcc
profile:
version: 11
cc: /usr/bin/gcc
cxx: /usr/bin/g++
- compiler: clang
profile:
version: 14
cc: /usr/bin/clang-14
cxx: /usr/bin/clang++-14
runs-on: [self-hosted, heavy]
container: thejohnfreeman/rippled-build-ubuntu:12e19cd9034b
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe push a new ubuntu 18.04 image with GCC-10?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 18.04?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we can test it into perpetuity?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I think there's a case to be made to pick one configuration to test on multiple versions of Ubuntu. I don't have an 18.04 image with everything available right now. Won't take me long to make one, but perhaps we can punt that to a future PR?

steps:
- name: checkout
uses: actions/checkout@v3
- name: install Ninja on Linux
if: matrix.generator == 'Ninja' && runner.os == 'Linux'
run: sudo apt install ninja-build
- name: install Ninja on OSX
if: matrix.generator == 'Ninja' && runner.os == 'macOS'
run: brew install ninja
- name: install nproc on OSX
if: runner.os == 'macOS'
run: brew install coreutils
- name: choose Python
uses: actions/setup-python@v3
with:
python-version: 3.9
- name: learn Python cache directory
id: pip-cache
run: |
sudo pip install --upgrade pip
echo "::set-output name=dir::$(pip cache dir)"
- name: restore Python cache directory
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-${{ hashFiles('.github/workflows/nix.yml') }}
- name: install Conan
run: pip install wheel 'conan~=1.52'
- name: check environment
run: |
echo ${PATH} | tr ':' '\n'
python --version
conan --version
cmake --version
env
- name: configure Conan
run: |
conan profile new default --detect
conan profile update settings.compiler.cppstd=20 default
thejohnfreeman marked this conversation as resolved.
Show resolved Hide resolved
- name: configure Conan on Linux
if: runner.os == 'Linux'
run: |
conan profile update settings.compiler=${{ matrix.compiler }} default
conan profile update settings.compiler.version=${{ matrix.profile.version }} default
conan profile update settings.compiler.libcxx=libstdc++11 default
- name: learn Conan cache directory
id: conan-cache
conan profile update env.CC=${{ matrix.profile.cc }} default
conan profile update env.CXX=${{ matrix.profile.cxx }} default
conan profile update conf.tools.build:compiler_executables='{"c": "${{ matrix.profile.cc }}", "cpp": "${{ matrix.profile.cxx }}"}' default
- name: checkout
uses: actions/checkout@v3
- name: build dependencies
run: |
echo "::set-output name=dir::$(conan config get storage.path)"
- name: restore Conan cache directory
uses: actions/cache@v2
mkdir .build
cd .build
conan install \
--output-folder . \
--build missing \
--settings build_type=${{ matrix.configuration }} \
..
- name: archive cache
run: tar -czf conan.tar -C ~/.conan .
- name: upload cache
uses: actions/upload-artifact@v3
with:
path: ${{ steps.conan-cache.outputs.dir }}
key: ${{ hashFiles('~/.conan/profiles/default', 'conanfile.py', 'external/rocksdb/*', '.github/workflows/nix.yml') }}
- name: export Snappy
run: conan export external/snappy snappy/1.1.9@
- name: install dependencies
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
path: conan.tar


test:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
- clang
configuration:
- Debug
- Release
cmake-args:
-
- "-Dunity=ON"
needs: dependencies
runs-on: [self-hosted, heavy]
container: thejohnfreeman/rippled-build-ubuntu:12e19cd9034b
env:
build_dir: .build
steps:
- name: download cache
uses: actions/download-artifact@v3
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
- name: extract cache
run: |
mkdir ${build_dir}
cd ${build_dir}
conan install .. --build missing --settings build_type=${{ matrix.configuration }}
- name: configure
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: check environment
run: |
cd ${build_dir}
cmake \
-G ${{ matrix.generator }} \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${{ matrix.configuration }} \
-Dassert=ON \
-Dcoverage=OFF \
-Dreporting=OFF \
-Dunity=OFF \
..
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
env
ls ~/.conan
- name: checkout
uses: actions/checkout@v3
- name: build
run: |
cmake --build ${build_dir} --target rippled --parallel $(nproc)
uses: ./.github/actions/build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
cmake-args: ${{ matrix.cmake-args }}
- name: test
run: |
${build_dir}/rippled --unittest --unittest-jobs $(nproc)