Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Merge in contents of build-nargo repository #2211

Merged
merged 28 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3c8a2bb
chore: verbatim copy of `build-nargo`
TomAFrench Aug 8, 2023
60d3597
chore: switch to yarn
TomAFrench Aug 8, 2023
97f0889
chore: remove unnecessary noir directory
TomAFrench Aug 8, 2023
23c1dbd
chore: disable publishing and nightly schedule
TomAFrench Aug 8, 2023
439ced2
chore: run release tests from correct directory
TomAFrench Aug 8, 2023
f2c9b89
chore: remove double checkout
TomAFrench Aug 8, 2023
15e9f93
chore: checkout noir repo into `./noir` repository
TomAFrench Aug 8, 2023
3faff53
chore: update path to cross config
TomAFrench Aug 8, 2023
a280b91
chore: re-enable publishing nightly releases
TomAFrench Aug 8, 2023
a2a55eb
chore: read GITHUB_REF properly
TomAFrench Aug 8, 2023
b9dab0a
chore: set up triggers for publishing
TomAFrench Aug 8, 2023
149a8c1
chore: add kev to assignees
TomAFrench Aug 8, 2023
410b0b0
chore: use actual workflow path
TomAFrench Aug 8, 2023
dccf5f6
chore: reenable reporting of nightly failures
TomAFrench Aug 8, 2023
cece5d3
chore: remove commented toml
TomAFrench Aug 8, 2023
33d55c7
chore: run workflow on merge queue
TomAFrench Aug 9, 2023
c656cda
chore: remove `./noir` subdirectory
TomAFrench Aug 9, 2023
7c6c84a
chore: run build to check that it's working
TomAFrench Aug 9, 2023
3f3ea92
chore: replace `NOIR_REPO_TOKEN` with `GITHUB_TOKEN`
TomAFrench Aug 9, 2023
1b0a067
chore: move ignored directory to repo root
TomAFrench Aug 9, 2023
92d7d00
chore: revert to using `workflow_dispatch`
TomAFrench Aug 9, 2023
4134588
chore: remove push trigger
TomAFrench Aug 9, 2023
1b4f0b1
chore: add `permissions` section
TomAFrench Aug 10, 2023
0d543e9
chore: remove auth token from cachix
TomAFrench Aug 10, 2023
543f77a
temp
TomAFrench Aug 10, 2023
68186fc
chore: list dual license in `package.json`s
TomAFrench Aug 10, 2023
66008ae
chore: remove todo
TomAFrench Aug 10, 2023
e458fd1
chore: remove `push:` trigger
TomAFrench Aug 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/Cross.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[build.env]
passthrough = [
"HOME",
"RUST_BACKTRACE",
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
"BARRETENBERG_BIN_DIR"
]
volumes = [
"HOME",
]
11 changes: 11 additions & 0 deletions .github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
title: "nightly build failed"
assignees: kobyhallx, phated, tomafrench, kevaundray
labels: bug
---

Something broke our nightly builds.

Check the [build]({{env.WORKFLOW_URL}}) workflow for details.

This issue was raised by the workflow `{{env.WORKFLOW_NAME}}`
356 changes: 356 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,356 @@
name: Publish Nargo

on:
workflow_dispatch:
# Allow pushing a manual nightly release
inputs:
tag:
description: The tag to build Nargo from (leave empty to build a nightly release from master)
required: false
publish:
description: Whether to publish the build artifacts
type: boolean
default: false
schedule:
# Run a nightly release at 2 AM UTC
- cron: "0 2 * * *"
push:
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
# release-please staging branch builds binaries as dry-run for release
branches:
- "release-please--branches--master--components--noir"

jobs:
build-barretenberg:
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ inputs.tag || env.GITHUB_REF }}

- name: Collect locked barretenberg rev
run: |
echo "BB_REV=$(jq -r .nodes.barretenberg.locked.rev ./flake.lock)" >> $GITHUB_ENV

- uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-22.11
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@v12
with:
name: barretenberg
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

# Upload does not work with symlinks, using this workaround:
# https://github.com/actions/upload-artifact/issues/92#issuecomment-1080347032
- name: Build barretenberg as libbarretenberg-wasm32
run: |
nix build "github:AztecProtocol/barretenberg/${{ env.BB_REV }}#wasm32"
echo "ARTIFACT_UPLOAD_PATH=$(readlink -f result)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ env.ARTIFACT_UPLOAD_PATH }}
retention-days: 3

# If we're performing a nightly build and it fails acceptance tests then raise an issue.
- name: Alert on nightly build failure
uses: JasonEtco/create-an-issue@v2
if: ${{ failure() && github.event_name == 'schedule' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_NAME: ${{ github.workflow }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
update_existing: true
filename: .github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

build-apple-darwin:
needs: [build-barretenberg]
runs-on: macos-latest
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
matrix:
target: [x86_64-apple-darwin, aarch64-apple-darwin]

steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ inputs.tag || env.GITHUB_REF }}
path: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

- name: Setup for Apple Silicon
if: matrix.target == 'aarch64-apple-darwin'
run: |
sudo xcode-select -s /Applications/Xcode_13.2.1.app/Contents/Developer/
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
echo "SDKROOT=$(xcrun -sdk macosx$(sw_vers -productVersion) --show-sdk-path)" >> $GITHUB_ENV
echo "MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx$(sw_vers -productVersion) --show-sdk-platform-version)" >> $GITHUB_ENV

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build environment and Compile
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo build --package nargo_cli --release --target ${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-${{ matrix.target }}.tar.gz
kevaundray marked this conversation as resolved.
Show resolved Hide resolved

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./noir/dist/*
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
retention-days: 3

- name: Test built artifact
if: matrix.target == 'x86_64-apple-darwin'
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/

cd release-tests
yarn install
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
yarn test

- name: Upload binaries to release tag
uses: svenstaro/upload-release-action@v2
if: ${{ inputs.publish || github.event_name == 'schedule' }}
with:
repo_name: noir-lang/noir
repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
file: ./noir/nargo-${{ matrix.target }}.tar.gz
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
asset_name: nargo-${{ matrix.target }}.tar.gz
overwrite: true
tag: ${{ inputs.tag || 'nightly' }} # This will fail if `inputs.tag` is not a tag (e.g. testing a branch)

# If we're performing a nightly build and it fails acceptance tests then raise an issue.
- name: Alert on nightly build failure
uses: JasonEtco/create-an-issue@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_NAME: ${{ github.workflow }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
update_existing: true
filename: ./noir/.github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

build-linux:
needs: [build-barretenberg]
runs-on: ubuntu-22.04
env:
CROSS_CONFIG: ${{ github.workspace }}/noir/.github/Cross.toml
strategy:
fail-fast: false
matrix:
target:
[
x86_64-unknown-linux-gnu,
x86_64-unknown-linux-musl,
aarch64-unknown-linux-gnu,
aarch64-unknown-linux-musl,
]

steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ inputs.tag || env.GITHUB_REF }}
path: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build Nargo
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo install cross --force --git https://github.com/cross-rs/cross
cross build --package nargo_cli --release --target=${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-${{ matrix.target }}.tar.gz

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./noir/dist/*
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
retention-days: 3

- name: Test built artifact
if: startsWith(matrix.target, 'x86_64-unknown-linux')
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/

cd release-tests
yarn install
yarn test

- name: Upload binaries to release tag
uses: svenstaro/upload-release-action@v2
if: ${{ inputs.publish || github.event_name == 'schedule' }}
with:
repo_name: noir-lang/noir
repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
file: ./noir/nargo-${{ matrix.target }}.tar.gz
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
asset_name: nargo-${{ matrix.target }}.tar.gz
overwrite: true
tag: ${{ inputs.tag || 'nightly' }} # This will fail if `inputs.tag` is not a tag (e.g. testing a branch)

# If we're performing a nightly build and it fails acceptance tests then raise an issue.
- name: Alert on nightly build failure
uses: JasonEtco/create-an-issue@v2
if: ${{ failure() && github.event_name == 'schedule' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_NAME: ${{ github.workflow }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
update_existing: true
filename: ./noir/.github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

build-windows:
needs: [build-barretenberg]
runs-on: windows-2022
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
matrix:
target: [x86_64-pc-windows-msvc]

steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ inputs.tag || env.GITHUB_REF }}
path: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build environment and Compile
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo build --package nargo_cli --release --target ${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo.exe ./dist/nargo.exe
7z a -tzip nargo-${{ matrix.target }}.zip ./dist/*

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./noir/dist/*
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
retention-days: 3

- name: Test built artifact
working-directory: ./noir
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
shell: powershell
run: |
cp ./target/${{ matrix.target }}/release/nargo.exe ~/.cargo/bin/

cd release-tests
yarn install
yarn test

- name: Upload binaries to release tag
uses: svenstaro/upload-release-action@v2
if: ${{ inputs.publish || github.event_name == 'schedule' }}
with:
repo_name: noir-lang/noir
repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
file: ./noir/nargo-${{ matrix.target }}.zip
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
asset_name: nargo-${{ matrix.target }}.zip
overwrite: true
tag: ${{ inputs.tag || 'nightly' }} # This will fail if `inputs.tag` is not a tag (e.g. testing a branch)

# If we're performing a nightly build and it fails acceptance tests then raise an issue.
- name: Alert on nightly build failure
uses: JasonEtco/create-an-issue@v2
if: ${{ failure() && github.event_name == 'schedule' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_NAME: ${{ github.workflow }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
update_existing: true
filename: ./noir/.github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
14 changes: 4 additions & 10 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,10 @@ jobs:
name: Build binaries
needs: [release-please]
if: ${{ needs.release-please.outputs.tag-name }}
runs-on: ubuntu-latest
steps:
- name: Dispatch to build-nargo
uses: benc-uk/workflow-dispatch@v1
with:
workflow: publish.yml
repo: noir-lang/build-nargo
ref: master
token: ${{ secrets.NOIR_REPO_TOKEN }}
inputs: '{ "noir-ref": "${{ needs.release-please.outputs.tag-name }}", "publish": true }'
uses: ./.github/workflows/publish.yml
with:
tag: ${{ needs.release-please.outputs.tag-name }}
publish: true
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

publish-wasm:
name: Publish noir_wasm package
Expand Down
Loading