Skip to content
This repository has been archived by the owner on Oct 8, 2024. It is now read-only.

Move x86 jobs to GHA #169

Merged
merged 8 commits into from
Nov 16, 2020
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@

if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
if [[ "$MB_PYTHON_VERSION" == "pypy3.6-7.3" ]]; then
# for https://foss.heptapod.net/pypy/pypy/-/issues/3229
# TODO remove when that is fixed
brew install tcl-tk
fi
# these cause a conflict with built webp and libtiff
brew remove --ignore-dependencies webp zstd xz libtiff
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This resolves #133 which reappears on GHA.

fi

if [[ "$MB_PYTHON_VERSION" == "pypy3.6-7.3" ]]; then
if [[ "$TRAVIS_OS_NAME" != "macos-latest" ]]; then
MB_ML_VER="2010"
DOCKER_TEST_IMAGE="multibuild/xenial_$PLAT"
else
MB_PYTHON_OSX_VER="10.9"
fi
fi

echo "::group::Install a virtualenv"
source multibuild/common_utils.sh
source multibuild/travis_steps.sh
# can't use default 7.3.1 on macOS due to https://foss.heptapod.net/pypy/pypy/-/issues/3229
LATEST_PP_7p3=7.3.2
pip install virtualenv
before_install
echo "::endgroup::"

echo "::group::Build wheel"
clean_code $REPO_DIR $BUILD_COMMIT
build_wheel $REPO_DIR $PLAT
ls -l "${GITHUB_WORKSPACE}/${WHEEL_SDIR}/"
echo "::endgroup::"

echo "::group::Test wheel"
install_run $PLAT
echo "::endgroup::"
104 changes: 104 additions & 0 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@

name: Wheels

on: [push, pull_request]

env:
REPO_DIR: Pillow
BUILD_DEPENDS: ""
TEST_DEPENDS: "pytest pytest-cov"
MACOSX_DEPLOYMENT_TARGET: "10.10"
WHEEL_SDIR: wheelhouse

jobs:
build:
name: ${{ matrix.python }} ${{ matrix.os-name }} ${{ matrix.platform }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ "ubuntu-16.04", "macos-latest" ]
python: [ "pypy3.6-7.3", "3.6", "3.7", "3.8", "3.9" ]
platform: [ "x86_64", "i686" ]
exclude:
- os: "macos-latest"
platform: "i686"
include:
- os: "macos-latest"
os-name: "osx"
- os: "ubuntu-16.04"
os-name: "xenial"
env:
BUILD_COMMIT: HEAD
PLAT: ${{ matrix.platform }}
MB_PYTHON_VERSION: ${{ matrix.python }}
TRAVIS_OS_NAME: ${{ matrix.os-name }}
steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Build Wheel
run: .github/workflows/build.sh
- uses: actions/upload-artifact@v2
with:
name: wheels
path: wheelhouse/*.whl
# Uncomment to get SSH access for testing
# - name: Setup tmate session
# if: failure()
# uses: mxschmitt/action-tmate@v3
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this action to be invaluable in debugging the macOS build, as I do not have any other way of using a macOS machine.


build-latest:
name: ${{ matrix.python }} ${{ matrix.os-name }} ${{ matrix.platform }} latest
runs-on: ${{ matrix.os }}
if: "!startsWith(github.ref, 'refs/tags/')"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest only runs on non-tagged builds as a second job. This requires duplicating the build matrix.

This is not as necessary on GHA as on Travis due to the speed difference, but I tried to match the Travis jobs as closely as possible for now.

I also put PyPy first to give it a head-start on macOS, it is by far the slowest job at ~15m.

strategy:
fail-fast: false
matrix:
os: [ "ubuntu-16.04", "macos-latest" ]
python: [ "pypy3.6-7.3", "3.6", "3.7", "3.8", "3.9" ]
platform: [ "x86_64", "i686" ]
exclude:
- os: "macos-latest"
platform: "i686"
include:
- os: "macos-latest"
os-name: "osx"
- os: "ubuntu-16.04"
os-name: "xenial"
env:
BUILD_COMMIT: master
PLAT: ${{ matrix.platform }}
MB_PYTHON_VERSION: ${{ matrix.python }}
TRAVIS_OS_NAME: ${{ matrix.os-name }}
steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Build Wheel
run: .github/workflows/build.sh
- uses: actions/upload-artifact@v2
with:
name: wheels-latest
path: wheelhouse/*.whl

release:
name: Create Release
runs-on: ubuntu-latest
if: "startsWith(github.ref, 'refs/tags/')"
needs: build
steps:
- uses: actions/download-artifact@v2
with:
name: wheels
- name: Upload Release
uses: fnkr/github-action-ghr@v1.3
env:
GHR_PATH: .
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Comment on lines +100 to +104
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "official" actions for this by GHA are no longer actively supported, actions/upload-release-asset#58 (comment). They also break badly if a release already exists, and don't support wildcards.

Testing, I have found this action to work well, the only issue is if a successful build is re-run for whatever reason, with it failing to upload duplicate filenames. The ${{ secrets.GITHUB_TOKEN }} is provided automatically.

184 changes: 0 additions & 184 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,84 +16,6 @@ services: docker

jobs:
include:
- name: "3.6 macOS"
os: osx
osx_image: xcode9.3
language: generic
env:
- MB_PYTHON_VERSION=3.6

- name: "3.6 Xenial"
os: linux
env:
- MB_PYTHON_VERSION=3.6
- name: "3.6 Xenial 32-bit"
os: linux
env:
- MB_PYTHON_VERSION=3.6
- PLAT=i686

- name: "3.7 macOS"
os: osx
osx_image: xcode9.3
language: generic
env:
- MB_PYTHON_VERSION=3.7
- name: "3.8 macOS"
os: osx
osx_image: xcode9.3
language: generic
env:
- MB_PYTHON_VERSION=3.8
- name: "3.9 macOS"
os: osx
osx_image: xcode9.3
language: generic
env:
- MB_PYTHON_VERSION=3.9

- name: "3.7 Xenial"
os: linux
env:
- MB_PYTHON_VERSION=3.7
- name: "3.7 Xenial 32-bit"
os: linux
env:
- MB_PYTHON_VERSION=3.7
- PLAT=i686
- name: "3.8 Xenial"
os: linux
env:
- MB_PYTHON_VERSION=3.8
- name: "3.8 Xenial 32-bit"
os: linux
env:
- MB_PYTHON_VERSION=3.8
- PLAT=i686
- name: "3.9 Xenial"
os: linux
env:
- MB_PYTHON_VERSION=3.9
- name: "3.9 Xenial 32-bit"
os: linux
env:
- MB_PYTHON_VERSION=3.9
- PLAT=i686

- name: "3.6 macOS PyPy"
os: osx
osx_image: xcode9.3
language: generic
env:
- MB_PYTHON_VERSION=pypy3.6-7.3
- MB_PYTHON_OSX_VER=10.9
- name: "3.6 Xenial 64-bit PyPy"
os: linux
env:
- MB_PYTHON_VERSION=pypy3.6-7.3
- MB_ML_VER=2010
- DOCKER_TEST_IMAGE=multibuild/xenial_{PLAT}

- name: "3.6 Xenial aarch64"
arch: arm64
env:
Expand Down Expand Up @@ -125,112 +47,6 @@ jobs:
- MB_PYTHON_VERSION=3.9
- DOCKER_TEST_IMAGE=multibuild/xenial_{PLAT}

- name: "3.6 macOS latest"
os: osx
osx_image: xcode9.3
language: generic
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.6
- LATEST="true"

- name: "3.6 Xenial latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.6
- LATEST="true"
- name: "3.6 Xenial 32-bit latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.6
- PLAT=i686
- LATEST="true"

- name: "3.7 macOS latest"
os: osx
osx_image: xcode9.3
language: generic
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.7
- LATEST="true"
- name: "3.8 macOS latest"
os: osx
osx_image: xcode9.3
language: generic
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.8
- LATEST="true"
- name: "3.9 macOS latest"
os: osx
osx_image: xcode9.3
language: generic
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.9
- LATEST="true"

- name: "3.7 Xenial latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.7
- LATEST="true"
- name: "3.7 Xenial 32-bit latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.7
- PLAT=i686
- LATEST="true"
- name: "3.8 Xenial latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.8
- LATEST="true"
- name: "3.8 Xenial 32-bit latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.8
- PLAT=i686
- LATEST="true"
- name: "3.9 Xenial latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.9
- LATEST="true"
- name: "3.9 Xenial 32-bit latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=3.9
- PLAT=i686
- LATEST="true"

- name: "3.6 macOS PyPy latest"
os: osx
osx_image: xcode9.3
language: generic
if: tag IS blank
env:
- MB_PYTHON_VERSION=pypy3.6-7.3
- MB_PYTHON_OSX_VER=10.9
- LATEST="true"
- name: "3.6 Xenial 64-bit PyPy latest"
os: linux
if: tag IS blank
env:
- MB_PYTHON_VERSION=pypy3.6-7.3
- MB_ML_VER=2010
- DOCKER_TEST_IMAGE=multibuild/xenial_{PLAT}
- LATEST="true"

- name: "3.6 Xenial aarch64 latest"
os: linux
arch: arm64
Expand Down
14 changes: 2 additions & 12 deletions config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,29 +52,19 @@ function pre_build {
CFLAGS="$CFLAGS -g -O2"
build_jpeg
CFLAGS=$ORIGINAL_CFLAGS

build_tiff
build_libpng
build_lcms2
build_openjpeg

if [ -n "$IS_OSX" ]; then
# Custom flags to allow building on OS X 10.10 and 10.11
build_giflib

ORIGINAL_CPPFLAGS=$CPPFLAGS
CPPFLAGS=""
fi
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found that GHA no longer seems to require #104.

CFLAGS="$CFLAGS -O3 -DNDEBUG"
build_libwebp
CFLAGS=$ORIGINAL_CFLAGS
if [ -n "$IS_OSX" ]; then
CPPFLAGS=$ORIGINAL_CPPFLAGS
fi

if [ -n "$IS_OSX" ]; then
# Custom freetype build
build_simple freetype $FREETYPE_VERSION https://download.savannah.gnu.org/releases/freetype tar.gz --with-harfbuzz=no
build_simple freetype $FREETYPE_VERSION https://download.savannah.gnu.org/releases/freetype tar.gz --with-harfbuzz=no --with-brotli=no
else
build_freetype
fi
Expand Down