Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy package both in Python 2 and Python 3 #671

Merged
merged 2 commits into from
May 22, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 86 additions & 124 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,189 +1,151 @@
# CircleCI 2.0 configuration file. See <https://circleci.com/docs/2.0/language-python/>.
version: 2

jobs:
checkout:
working_directory: ~/openfisca-core
build_python2:
docker:
- image: python:2.7.14

steps:
- checkout

- restore_cache:
key: v1-py2-{{ checksum "setup.py" }}

- run:
name: Fetch remote refs
command: git fetch
name: Create a virtualenv
command: |
mkdir -p /tmp/venv/openfisca_core
virtualenv /tmp/venv/openfisca_core
echo "source /tmp/venv/openfisca_core/bin/activate" >> $BASH_ENV

- run:
name: Install dependencies
command: |
pip install --upgrade pip twine wheel
pip install --editable .[test] --upgrade
# pip install --editable git+https://github.com/openfisca/country-template.git@BRANCH_NAME#egg=OpenFisca-Country-Template # use a specific branch of OpenFisca-Country-Template
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be Core.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenFisca Country Template is a test dependency of Core. This line allows you to run the tests with a specific branch of the country template.

This is useful when working and breaking changes that imply editions both on core and country template.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah good point!


- save_cache:
key: v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
key: v1-py2-{{ checksum "setup.py" }}
paths:
- .
- ~/.ssh/known_hosts
python3:
working_directory: ~/openfisca-core
docker:
- image: python:3.6

steps:
- checkout
- /tmp/venv/openfisca_core

- run:
name: Fetch remote refs
command: git fetch
name: Run tests
command: make test

- save_cache:
key: v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
paths:
- .
- ~/.ssh/known_hosts
- run:
name: Check version number has been properly updated
command: |
git fetch
.circleci/is-version-number-acceptable.sh

dependencies:
working_directory: ~/openfisca-core
deploy_python2:
docker:
- image: python:2.7.14
environment:
PYPI_USERNAME: openfisca-bot # Edit this value to replace it by your Pypi username
# PYPI_PASSWORD: this value is set in CircleCI's web interface; do not set it here, it is a secret!

steps:
- checkout

- restore_cache:
keys:
- v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
key: v1-py2-{{ checksum "setup.py" }}

- run:
name: Create virtualenv
command: |
mkdir -p /tmp/venv/openfisca-core
virtualenv /tmp/venv/openfisca-core
name: Check for functional changes
command: if .circleci/detect-functional-changes.sh ; then circleci step halt ; fi

# pip >= 8.0 needed to be compatible with "manylinux" wheels, used by numpy >= 1.11
- run:
name: Install dependencies
name: Upload a Python package to Pypi
command: |
. /tmp/venv/openfisca-core/bin/activate
pip install --upgrade pip twine wheel
pip install .[test] --upgrade
source /tmp/venv/openfisca_core/bin/activate
.circleci/publish-python-package.sh

# Uncomment and adapt the next line to use a particular feature branch of OpenFisca-Country-Template to run Circle CI tests
# - run:
# name: Install a particular feature branch of OpenFisca-Country-Template
# command: |
# . /tmp/venv/openfisca-core/bin/activate
# pip install --editable git+https://github.com/openfisca/country-template.git@BRANCH_NAME#egg=OpenFisca-Country-Template
- run:
name: Publish a git tag
command: .circleci/publish-git-tag.sh

- save_cache:
key: v1-dependencies-{{ .Environment.CIRCLE_SHA1 }}
paths:
- /tmp/venv/openfisca-core
dependencies_python3:
working_directory: ~/openfisca-core
build_python3:
docker:
- image: python:3.6

steps:
- checkout

- restore_cache:
keys:
- v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
key: v1-py3-{{ checksum "setup.py" }}

- run:
name: Create a virtualenv
command: |
mkdir -p /tmp/venv/openfisca_core
python -m venv /tmp/venv/openfisca_core
echo "source /tmp/venv/openfisca_core/bin/activate" >> $BASH_ENV

# pip >= 8.0 needed to be compatible with "manylinux" wheels, used by numpy >= 1.11
- run:
name: Install dependencies
command: |
pip install --upgrade pip twine wheel

# Uncomment and adapt the next line to use a particular feature branch of OpenFisca-Country-Template to run Circle CI tests
# - run:
# name: Install a particular feature branch of OpenFisca-Country-Template
# command: |
# . /tmp/venv/openfisca-core/bin/activate
# pip install --editable git+https://github.com/openfisca/country-template.git@BRANCH_NAME#egg=OpenFisca-Country-Template
pip install --editable .[test] --upgrade
# pip install --editable git+https://github.com/openfisca/country-template.git@BRANCH_NAME#egg=OpenFisca-Country-Template # use a specific branch of OpenFisca-Country-Template
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be Core

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See reply above


- save_cache:
key: v1-dependencies-{{ .Environment.CIRCLE_SHA1 }}
key: v1-py3-{{ checksum "setup.py" }}
paths:
- /tmp/venv/openfisca-core
tests:
working_directory: ~/openfisca-core
docker:
- image: python:2.7.14

steps:
- restore_cache:
keys:
- v1-checkout-{{ .Environment.CIRCLE_SHA1 }}

- restore_cache:
keys:
- v1-dependencies-{{ .Environment.CIRCLE_SHA1 }}
- /tmp/venv/openfisca_core

- run:
name: Run tests
command: |
. /tmp/venv/openfisca-core/bin/activate
make test
. check-version-bump.sh
tests_python3:
working_directory: ~/openfisca-core
docker:
- image: python:3.6

steps:
- restore_cache:
keys:
- v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
command: make test

- restore_cache:
keys:
- v1-dependencies-{{ .Environment.CIRCLE_SHA1 }}
- run:
name: Run tests
name: Check version number has been properly updated
command: |
pip install --upgrade pip twine wheel
pip install --editable .[test] --upgrade
make test
git fetch
.circleci/is-version-number-acceptable.sh

deploy:
working_directory: ~/openfisca-core
deploy_python3:
docker:
- image: python:2.7.14
environment:
PYPI_USERNAME: openfisca-bot
# PYPI_PASSWORD: this value is set in CircleCI's web interface; do not set it here, it is a secret!
- image: python:3.6
environment:
PYPI_USERNAME: openfisca-bot
# PYPI_PASSWORD: this value is set in CircleCI's web interface; do not set it here, it is a secret!

steps:
- restore_cache:
keys:
- v1-checkout-{{ .Environment.CIRCLE_SHA1 }}
- checkout

- restore_cache:
keys:
- v1-dependencies-{{ .Environment.CIRCLE_SHA1 }}
key: v1-py3-{{ checksum "setup.py" }}

- run:
name: Check for functional changes
command: if .circleci/detect-functional-changes.sh ; then circleci step halt ; fi

- run:
name: Deploy (if version bump)
name: Upload a Python package to Pypi
command: |
. /tmp/venv/openfisca-core/bin/activate
. deploy-if-version-bump.sh
source /tmp/venv/openfisca_core/bin/activate
.circleci/publish-python-package.sh

workflows:
version: 2
openfisca-core:
build_and_deploy:
jobs:
- checkout
- dependencies:
requires:
- checkout
- tests:
requires:
- dependencies
- python3:
requires:
- tests
- dependencies_python3:
- build_python2
- build_python3
- deploy_python2:
requires:
- python3
- tests_python3:
requires:
- dependencies_python3
- deploy:
- build_python2
- build_python3
filters:
branches:
only: master
- deploy_python3:
requires:
- tests
- build_python2
- build_python3
filters:
branches:
only: master
12 changes: 12 additions & 0 deletions .circleci/detect-functional-changes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#! /usr/bin/env bash

IGNORE_DIFF_ON="README.md CONTRIBUTING.md .gitignore .circleci/* .github/*"

last_tagged_commit=`git describe --tags --abbrev=0 --first-parent` # --first-parent ensures we don't follow tags not published in master through an unlikely intermediary merge commit

if git diff-index --name-only --exit-code $last_tagged_commit -- . `echo " $IGNORE_DIFF_ON" | sed 's/ / :(exclude)/g'`
then echo "No functional changes detected."
else
echo "The functional files above were changed."
exit 1
fi
33 changes: 33 additions & 0 deletions .circleci/is-version-number-acceptable.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#! /usr/bin/env bash

if [[ $CIRCLE_BRANCH == master ]]
then
echo "No need for a version check on master."
exit 0
fi

if $(dirname "$BASH_SOURCE")/detect-functional-changes.sh
then
echo "No need for a version update."
exit 0
fi

current_version=`python setup.py --version`

if git rev-parse --verify --quiet $current_version
then
echo "Version $current_version already exists in commit:"
git --no-pager log -1 $current_version
echo
echo "Update the version number in setup.py before merging this branch into master."
echo "Look at the CONTRIBUTING.md file to learn how the version number should be updated."
exit 1
fi

if ! $(dirname "$BASH_SOURCE")/detect-functional-changes.sh | grep --quiet CHANGELOG.md
then
echo "CHANGELOG.md has not been modified, while functional changes were made."
echo "Explain what you changed before merging this branch into master."
echo "Look at the CONTRIBUTING.md file to learn how to write the changelog."
exit 2
fi
4 changes: 4 additions & 0 deletions .circleci/publish-git-tag.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#! /usr/bin/env bash

git tag `python setup.py --version`
git push --tags # update the repository version
4 changes: 4 additions & 0 deletions .circleci/publish-python-package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#! /usr/bin/env bash

python setup.py bdist_wheel # build this package in the dist directory
twine upload dist/* --username $PYPI_USERNAME --password $PYPI_PASSWORD # publish
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

### 23.1.2 [#671](https://github.com/openfisca/openfisca-core/pull/671)

- Minor technical improvement
- Publish both Python 2 and Python 3 version on Pypi in CircleCI

### 23.1.1 [#661](https://github.com/openfisca/openfisca-core/pull/661)

- Fixup Python 3 compatibility
Expand Down
20 changes: 0 additions & 20 deletions check-version-bump.sh

This file was deleted.

12 changes: 0 additions & 12 deletions deploy-if-version-bump.sh

This file was deleted.

2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

setup(
name = 'OpenFisca-Core',
version = '23.1.1',
version = '23.1.2',
author = 'OpenFisca Team',
author_email = 'contact@openfisca.fr',
classifiers = [
Expand Down