Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to Pixi for development / CI and hatchling for build system #6775

Merged
merged 138 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from 82 commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
d265055
Automatic extract of dependencies from setup.py
hoxbro Apr 20, 2024
01e8786
Clean up dependencies, align with HoloViews
hoxbro Apr 20, 2024
0cfa04c
Add tasks to tests
hoxbro Apr 20, 2024
31cc42c
Run pixi install and fixes to make it work
hoxbro Apr 20, 2024
b7d4b18
Run test and lint tasks
hoxbro Apr 20, 2024
f071b86
Update pixi.toml with docs steps
hoxbro Apr 20, 2024
b216be9
Small updates to dependencies
hoxbro Apr 20, 2024
98558a5
Move project stuff to pyproject.toml
hoxbro Apr 20, 2024
db0183f
Add initial hatch setting to pyproject.toml
hoxbro Apr 20, 2024
26af6a1
Add nodejs and fix scripts path
hoxbro Apr 20, 2024
68c6964
Add build hook script
hoxbro Apr 20, 2024
6f2996e
Update validate js version
hoxbro Apr 20, 2024
7a86621
Add pip
hoxbro Apr 20, 2024
8a8afd3
Sort dependencies in pyproject.toml and pixi.toml
hoxbro Apr 21, 2024
0643f99
Small updates to pyproject.toml
hoxbro Apr 21, 2024
20bf27d
Remove tqdm and markdown for build-system
hoxbro Apr 21, 2024
f2535d8
Add shared-data for jupyter-config
hoxbro Apr 21, 2024
f9a25e6
Add unfixable rules for ruff
hoxbro Apr 21, 2024
df29373
Start enabling build
hoxbro Apr 21, 2024
2d86fd2
Add boilerplate __version__
hoxbro Apr 21, 2024
cc8de67
Add isort setting to pyproject
hoxbro Apr 21, 2024
3bb2d8c
Remove setup.cfg and setup.py
hoxbro Apr 21, 2024
9332c68
Move conda build to scripts folder
hoxbro Apr 21, 2024
362145d
Update conda meta
hoxbro Apr 21, 2024
1ff08d6
Add dist to force-include
hoxbro Apr 21, 2024
575580e
add license back
hoxbro Apr 21, 2024
fcd30a3
Update hatch-build
hoxbro Apr 21, 2024
d623bf5
Add PANEL_LITE back to install task
hoxbro Apr 22, 2024
e9d9705
Add some information + colors to build step
hoxbro Apr 22, 2024
d7bdcbf
Update to python-build
hoxbro Apr 22, 2024
c273521
enable build-pyodide
hoxbro Apr 22, 2024
f1cec0c
Small changes to test.yaml
hoxbro Apr 22, 2024
4989ddb
Update global settings for test.yaml workflow
hoxbro Apr 22, 2024
4e33fdf
Initial update to unit test
hoxbro Apr 22, 2024
750eb08
Initial update to UI test
hoxbro Apr 22, 2024
97462cf
Initial update to core test
hoxbro Apr 22, 2024
7c9d5d2
Small updates to test workflow
hoxbro Apr 22, 2024
124bb10
Add coverage environment variable
hoxbro Apr 22, 2024
05ee167
Remove tox.ini
hoxbro Apr 22, 2024
d2bf8cf
Remove dodo.py
hoxbro Apr 22, 2024
b50d2cf
Create the .uicoverage file in CI
hoxbro Apr 22, 2024
d05c1a9
First round of fixes
hoxbro Apr 22, 2024
1551fb7
Move test docs up
hoxbro Apr 22, 2024
1a51d76
Fix docs error
hoxbro Apr 22, 2024
a4c01c2
Add opengl to install task
hoxbro Apr 22, 2024
d2ed29e
Comment out validation check for now.
hoxbro Apr 22, 2024
af6d642
Add validate js version back
hoxbro Apr 22, 2024
e153669
Add wait for failing ui test
hoxbro Apr 22, 2024
24127de
Merge branch 'main' into pixi
hoxbro Apr 22, 2024
9c62914
Only mark jupyter test
hoxbro Apr 22, 2024
6bccb82
Stabalize test_icon.py flaky test
hoxbro Apr 22, 2024
28912bc
Try something with dynamic feed
hoxbro Apr 22, 2024
a3f450a
Update ui tests
hoxbro Apr 22, 2024
2f80d5c
Update build
hoxbro Apr 22, 2024
e671561
Only install if needed, try npm and cdn
hoxbro Apr 22, 2024
c8ddf6c
wait until for flaky tabulator test
hoxbro Apr 22, 2024
73b426c
More wait_until in test_feed
hoxbro Apr 22, 2024
7d7b958
Small misc changes
hoxbro Apr 23, 2024
0eea092
Move some packages from test-core to test
hoxbro Apr 23, 2024
75fbc8a
Use shell instead pixi run for every command
hoxbro Apr 23, 2024
2e7a86e
Merge branch 'main' into pixi
hoxbro Apr 23, 2024
2c4873f
Update build cdn
hoxbro Apr 23, 2024
e2a4c33
Add tracing back to ui test workflow
hoxbro Apr 23, 2024
db9153f
Kill jupyter lab for Windows
hoxbro Apr 23, 2024
26a7ff9
Format docs
hoxbro Apr 23, 2024
8c739b0
Update docs
hoxbro Apr 23, 2024
c6662f2
Switch to build and publish for docs
hoxbro Apr 23, 2024
66e2d88
Update Stop Jupyter Lab step
hoxbro Apr 23, 2024
d29423e
Fix spelling mistake...
hoxbro Apr 23, 2024
082d5dd
Add pixi-lock to docs build
hoxbro Apr 23, 2024
766211f
Remove test and uncomment upload parts for docs
hoxbro Apr 24, 2024
5f7161f
Bump peaceiris/action
hoxbro Apr 24, 2024
73c3664
Bump action version in test.yaml
hoxbro Apr 24, 2024
2ccccd0
Add tqdm to dependencies
hoxbro Apr 24, 2024
c0e71ac
Downgrade for wait-on-action
hoxbro Apr 24, 2024
78d9432
Only change the name of upload artifact
hoxbro Apr 24, 2024
9aff7f5
Add page to wait_for_server
hoxbro Apr 24, 2024
ea84488
Add task docs-server
hoxbro Apr 24, 2024
4e9c987
Make private tasks start with x
hoxbro Apr 24, 2024
82de22c
More wait_until
hoxbro Apr 24, 2024
4e104c1
Remove empty fixtures file
hoxbro Apr 24, 2024
93bc5e7
Remove MANIFEST.in
hoxbro Apr 24, 2024
22aaadb
Add nightly lock workflow
hoxbro Apr 24, 2024
c1840c7
Add wait_untils
hoxbro Apr 24, 2024
fee2ab9
Merge branch 'main' into pixi
hoxbro Apr 24, 2024
161fbb5
Bump param version
hoxbro Apr 24, 2024
fea4644
Pin ipyleaflet for now
hoxbro Apr 24, 2024
cca0155
Remove pin again
hoxbro Apr 24, 2024
de32136
Fix warning of class_ in param.List
hoxbro Apr 24, 2024
596d4e0
Try to reduce the number of items in dynamic_objects
hoxbro Apr 24, 2024
7b919e4
Reduce number of items
hoxbro Apr 25, 2024
c3e21d4
Wait for flaky tests
hoxbro Apr 25, 2024
f2cc4ed
Remove doc pins (to test out)
hoxbro Apr 25, 2024
d8c888d
Fix test and add wait_locator
hoxbro Apr 25, 2024
ae8ec00
Revert changes in this PR
hoxbro Apr 25, 2024
03420f7
add kwargs
hoxbro Apr 25, 2024
b162f04
Add some atol to test_gridstack
hoxbro Apr 25, 2024
7976bc4
Add atol to other gridstack function
hoxbro Apr 25, 2024
01affe7
Use macos-latest as they use mac-14 for it
hoxbro Apr 25, 2024
c3aa91f
Use runner.os for ui upload
hoxbro Apr 25, 2024
efbae0b
Pin nodejs
hoxbro Apr 29, 2024
5efa1cd
Update message with Bokeh PR
hoxbro Apr 29, 2024
2b5740b
Merge branch 'main' into pixi
hoxbro Apr 30, 2024
baf7921
Remove wait_locator as test would still fail
hoxbro Apr 30, 2024
fcd4600
Make private tasks
hoxbro May 1, 2024
7a26528
Add build-npm
hoxbro May 1, 2024
f684c06
Lint toml
hoxbro May 1, 2024
9dd30b8
Split dry-run to another step
hoxbro May 1, 2024
c7024cd
Update channel ordering
hoxbro May 2, 2024
f279f3e
Add timeout to nightly_lock workflow
hoxbro May 2, 2024
dd26308
Merge branch 'main' into pixi
hoxbro May 3, 2024
97383d6
Update optional dependencies
hoxbro May 3, 2024
d018928
Add examples to sdist
hoxbro May 3, 2024
59eb527
Add S3 upload to pixi nightly lock
hoxbro May 3, 2024
024612c
Make nighly lock upload a zip
hoxbro May 4, 2024
873e1f0
conftest update
hoxbro May 5, 2024
a37358d
Update wait-on-action
hoxbro May 5, 2024
f57738f
Update docs (mainly c/p from HoloViews)
hoxbro May 7, 2024
d23cab9
Split out custom models to own page
hoxbro May 7, 2024
1cbace9
Add lxml back
hoxbro May 7, 2024
b9d4865
Add more custom.css colors
hoxbro May 7, 2024
1fe8b89
Merge branch 'main' into pixi
hoxbro May 7, 2024
d8a1be7
Remove setuptools legacy from binder start
hoxbro May 7, 2024
682007f
Small doc fixes
hoxbro May 7, 2024
1240389
add comment about announcement banner
hoxbro May 7, 2024
2f4de71
Add --cov-append
hoxbro May 8, 2024
c60fdd9
Remove xyzservices from dependencies
hoxbro May 8, 2024
90d0034
Merge branch 'main' into pixi
hoxbro May 10, 2024
48b16f6
remove wrong setting
hoxbro May 10, 2024
ef735d7
Try pin IPython for UI tests
hoxbro May 10, 2024
4862f1b
Remove ipython pin
hoxbro May 10, 2024
b7cac3d
Sort colors
hoxbro May 11, 2024
e638a1e
Remove shell from build
hoxbro May 12, 2024
9f08dce
Merge branch 'main' into pixi
hoxbro May 13, 2024
355d67b
Move hatch_build.py to root directory
hoxbro May 13, 2024
a4faf34
Move custom models to markdown file
hoxbro May 13, 2024
1398221
Fix root directory for hatch_build.py
hoxbro May 13, 2024
7ab9403
Add hook back in Pyproject.toml
hoxbro May 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 31 additions & 58 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ env:
PYTHON_VERSION: "3.11"
NODE_VERSION: "20"
MPLBACKEND: "Agg"
SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"

jobs:
waiting_room:
Expand All @@ -33,33 +32,29 @@ jobs:
steps:
- run: echo "All builds have finished, have been approved, and ready to publish"

pixi_lock:
name: Pixi lock
runs-on: ubuntu-latest
steps:
- uses: holoviz-dev/holoviz_tasks/pixi_lock@pixi

conda_build:
name: Build Conda
needs: [pixi_lock]
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: "100"
- name: Fetch unshallow
run: git fetch --prune --tags --unshallow -f
- uses: conda-incubator/setup-miniconda@v3
with:
miniconda-version: "latest"
- uses: actions/setup-node@v4
- uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
with:
node-version: ${{ env.NODE_VERSION }}
- name: conda setup
run: |
conda install -y conda-build build
environments: "build"
install: false
download-data: false
- name: conda build
run: |
source ./scripts/build_conda.sh
echo "CONDA_FILE="$CONDA_PREFIX/conda-bld/noarch/$PACKAGE-$VERSION-py_0.tar.bz2"" >> $GITHUB_ENV
run: pixi run -e build build-conda
- uses: actions/upload-artifact@v4
if: always()
with:
name: conda
path: ${{ env.CONDA_FILE }}
path: dist/*tar.bz2
if-no-files-found: error

conda_publish:
Expand Down Expand Up @@ -93,24 +88,16 @@ jobs:

pip_build:
name: Build PyPI
needs: [pixi_lock]
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: "100"
- name: Fetch unshallow
run: git fetch --prune --tags --unshallow -f
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/setup-node@v4
- uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install build
run: |
python -m pip install build
environments: "build"
install: false
download-data: false
- name: Build package
run: python -m build .
run: pixi run -e build build-pip
- uses: actions/upload-artifact@v4
if: always()
with:
Expand Down Expand Up @@ -154,23 +141,15 @@ jobs:

npm_build:
name: Build NPM
needs: [pixi_lock]
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: "100"
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/setup-node@v4
- uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
with:
node-version: ${{ env.NODE_VERSION }}
- name: Fetch unshallow
run: git fetch --prune --tags --unshallow -f
- name: package install
run: |
python -m pip install -ve .
environments: "build"
download-data: false
- name: npm build
shell: pixi run -e build bash -el {0}
run: |
cd ${{ env.PACKAGE }}
TARBALL=$(npm pack .)
Expand Down Expand Up @@ -216,26 +195,20 @@ jobs:

cdn_build:
name: Build CDN
needs: [pixi_lock]
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: "100"
- uses: actions/setup-python@v5
- uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Fetch unshallow
run: git fetch --prune --tags --unshallow -f
environments: "build"
install: false
download-data: false
- name: build pyodide wheels
run: |
python -m pip install packaging
python ./scripts/build_pyodide_wheels.py --verify-clean
pixi run build-pyodide --verify-clean
- name: build CDN
run: |
python -m pip install -ve .
pixi run build-pip
- uses: actions/upload-artifact@v4
if: always()
with:
Expand Down
123 changes: 58 additions & 65 deletions .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,86 +3,79 @@ name: docs
on:
push:
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'
- 'v[0-9]+.[0-9]+.[0-9]+a[0-9]+'
- 'v[0-9]+.[0-9]+.[0-9]+b[0-9]+'
- 'v[0-9]+.[0-9]+.[0-9]+rc[0-9]+'
- "v[0-9]+.[0-9]+.[0-9]+"
- "v[0-9]+.[0-9]+.[0-9]+a[0-9]+"
- "v[0-9]+.[0-9]+.[0-9]+b[0-9]+"
- "v[0-9]+.[0-9]+.[0-9]+rc[0-9]+"
workflow_dispatch:
inputs:
target:
description: 'Site to build and deploy'
description: "Site to build and deploy"
type: choice
options:
- dev
- main
- dryrun
- dev
- main
- dryrun
required: true
default: dryrun
schedule:
- cron: '0 19 * * SUN'
- cron: "0 19 * * SUN"

defaults:
run:
shell: bash -el {0}

env:
MPLBACKEND: "Agg"
MOZ_HEADLESS: 1
DISPLAY: ":99.0"
PANEL_IPYWIDGET: 1

jobs:
build_docs:
name: Documentation
runs-on: 'macos-14'
pixi_lock:
name: Pixi lock
runs-on: ubuntu-latest
steps:
- uses: holoviz-dev/holoviz_tasks/pixi_lock@pixi

docs_build:
name: Build Documentation
needs: [pixi_lock]
runs-on: "macos-14"
timeout-minutes: 180
defaults:
run:
shell: bash -l {0}
env:
DESC: "Documentation build"
MPLBACKEND: "Agg"
SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"
MOZ_HEADLESS: 1
DISPLAY: ":99.0"
PANEL_IPYWIDGET: 1
outputs:
tag: ${{ steps.vars.outputs.tag }}
steps:
- uses: holoviz-dev/holoviz_tasks/install@v0
- uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
with:
name: doc_build
python-version: "3.11"
channels: pyviz/label/dev,conda-forge,bokeh,nodefaults
conda-update: 'true'
nodejs: true
nodejs-version: 20
# Remove when all examples tools can be installed on 3.10
envs: -o examples -o doc -o build
cache: true
environments: docs
opengl: true
- name: doit develop_install
if: steps.install.outputs.cache-hit != 'true'
run: |
conda activate test-environment
pip install --use-deprecated=legacy-resolver pyecharts ipywidgets_bokeh==1.5.0
- name: bokeh sampledata
run: |
conda activate test-environment
bokeh sampledata
- name: refmanual
run: |
conda activate test-environment
python ./doc/generate_modules.py panel -d ./doc/api -n panel -e tests
- name: build docs
run: |
conda activate test-environment
python scripts/gallery/convert_gallery.py
nbsite build --what=html --output=builtdocs --org holoviz --project-name panel
cp -r ./panel/dist ./builtdocs/panel_dist
- name: convert gallery
run: |
conda activate test-environment
panel convert examples/gallery/*.ipynb doc/how_to/*/examples/*.md --to pyodide-worker --out ./builtdocs/pyodide/ --pwa --index --requirements doc/pyodide_dependencies.json
- name: git status and git diff
run: |
git status
git diff
- name: Set and echo git ref
- name: Build documentation
run: pixi run -e docs docs-build
- uses: actions/upload-artifact@v4
if: always()
with:
name: docs
if-no-files-found: error
path: builtdocs
- name: Set output
id: vars
run: echo "tag=${GITHUB_REF#refs/*/}" >> $GITHUB_OUTPUT

docs_publish:
name: Publish Documentation
runs-on: "ubuntu-latest"
needs: [docs_build]
steps:
- uses: actions/download-artifact@v4
with:
name: docs
path: builtdocs/
- name: Set output
id: vars
run: |
echo "Deploying from ref ${GITHUB_REF#refs/*/}"
echo "tag=${GITHUB_REF#refs/*/}" >> $GITHUB_OUTPUT
run: echo "tag=${{ needs.docs_build.outputs.tag }}" >> $GITHUB_OUTPUT
- name: Deploy dev
uses: peaceiris/actions-gh-pages@v3
uses: peaceiris/actions-gh-pages@v4
if: |
(github.event_name == 'workflow_dispatch' && github.event.inputs.target == 'dev') ||
(github.event_name == 'push' && (contains(steps.vars.outputs.tag, 'a') || contains(steps.vars.outputs.tag, 'b') || contains(steps.vars.outputs.tag, 'rc')))
Expand All @@ -95,7 +88,7 @@ jobs:
if: |
(github.event_name == 'workflow_dispatch' && github.event.inputs.target == 'main') ||
(github.event_name == 'push' && !(contains(steps.vars.outputs.tag, 'a') || contains(steps.vars.outputs.tag, 'b') || contains(steps.vars.outputs.tag, 'rc')))
uses: peaceiris/actions-gh-pages@v3
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./builtdocs
Expand Down
Loading
Loading