Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Merge in contents of build-nargo repository #2211

Merged
merged 28 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3c8a2bb
chore: verbatim copy of `build-nargo`
TomAFrench Aug 8, 2023
60d3597
chore: switch to yarn
TomAFrench Aug 8, 2023
97f0889
chore: remove unnecessary noir directory
TomAFrench Aug 8, 2023
23c1dbd
chore: disable publishing and nightly schedule
TomAFrench Aug 8, 2023
439ced2
chore: run release tests from correct directory
TomAFrench Aug 8, 2023
f2c9b89
chore: remove double checkout
TomAFrench Aug 8, 2023
15e9f93
chore: checkout noir repo into `./noir` repository
TomAFrench Aug 8, 2023
3faff53
chore: update path to cross config
TomAFrench Aug 8, 2023
a280b91
chore: re-enable publishing nightly releases
TomAFrench Aug 8, 2023
a2a55eb
chore: read GITHUB_REF properly
TomAFrench Aug 8, 2023
b9dab0a
chore: set up triggers for publishing
TomAFrench Aug 8, 2023
149a8c1
chore: add kev to assignees
TomAFrench Aug 8, 2023
410b0b0
chore: use actual workflow path
TomAFrench Aug 8, 2023
dccf5f6
chore: reenable reporting of nightly failures
TomAFrench Aug 8, 2023
cece5d3
chore: remove commented toml
TomAFrench Aug 8, 2023
33d55c7
chore: run workflow on merge queue
TomAFrench Aug 9, 2023
c656cda
chore: remove `./noir` subdirectory
TomAFrench Aug 9, 2023
7c6c84a
chore: run build to check that it's working
TomAFrench Aug 9, 2023
3f3ea92
chore: replace `NOIR_REPO_TOKEN` with `GITHUB_TOKEN`
TomAFrench Aug 9, 2023
1b0a067
chore: move ignored directory to repo root
TomAFrench Aug 9, 2023
92d7d00
chore: revert to using `workflow_dispatch`
TomAFrench Aug 9, 2023
4134588
chore: remove push trigger
TomAFrench Aug 9, 2023
1b4f0b1
chore: add `permissions` section
TomAFrench Aug 10, 2023
0d543e9
chore: remove auth token from cachix
TomAFrench Aug 10, 2023
543f77a
temp
TomAFrench Aug 10, 2023
68186fc
chore: list dual license in `package.json`s
TomAFrench Aug 10, 2023
66008ae
chore: remove todo
TomAFrench Aug 10, 2023
e458fd1
chore: remove `push:` trigger
TomAFrench Aug 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/Cross.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[build]
# pre-build = [
# "dpkg --add-architecture $CROSS_DEB_ARCH",
# "apt-get update && apt-get install --assume-yes xz-utils pkgconfig build-essential cmake openssl libssl-dev libomp-dev libssl-dev:$CROSS_DEB_ARCH libomp-dev:$CROSS_DEB_ARCH",
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
# ]

[build.env]
passthrough = [
"HOME",
"RUST_BACKTRACE",
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
"BARRETENBERG_BIN_DIR"
]
volumes = [
"HOME",
]
11 changes: 11 additions & 0 deletions .github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
title: "nightly build failed"
assignees: kobyhallx, phated, tomafrench
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
labels: bug
---

Something broke our nightly builds.

Check the [build]({{env.WORKFLOW_URL}}) workflow for details.

This issue was raised by the workflow `{{env.WORKFLOW_NAME}}`
352 changes: 352 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,352 @@
name: Publish Nargo

on:
workflow_dispatch:
inputs:
noir-ref:
description: The noir reference to checkout
required: false
publish:
description: Whether to publish the build artifacts
type: boolean
default: false
# schedule:
# - cron: "0 2 * * *" # run at 2 AM UTC
push:
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

jobs:
build-barretenberg:
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Checkout Noir repo
uses: actions/checkout@v3
with:
ref: ${{ inputs.noir-ref || 'master' }}

- name: Collect locked barretenberg rev
run: |
echo "BB_REV=$(jq -r .nodes.barretenberg.locked.rev ./flake.lock)" >> $GITHUB_ENV

- uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-22.11
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@v12
with:
name: barretenberg
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

# Upload does not work with symlinks, using this workaround:
# https://github.com/actions/upload-artifact/issues/92#issuecomment-1080347032
- name: Build barretenberg as libbarretenberg-wasm32
run: |
nix build "github:AztecProtocol/barretenberg/${{ env.BB_REV }}#wasm32"
echo "ARTIFACT_UPLOAD_PATH=$(readlink -f result)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ env.ARTIFACT_UPLOAD_PATH }}
retention-days: 3

# If we're performing a nightly build and it fails acceptance tests then raise an issue.
- name: Alert on nightly build failure
uses: JasonEtco/create-an-issue@v2
if: ${{ failure() && github.event_name == 'schedule' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_NAME: ${{ github.workflow }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
update_existing: true
filename: .github/NIGHTLY_BUILD_FAILURE.md
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

build-apple-darwin:
needs: [build-barretenberg]
runs-on: macos-latest
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
matrix:
target: [x86_64-apple-darwin, aarch64-apple-darwin]

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Checkout Noir repo
uses: actions/checkout@v3
with:
ref: ${{ inputs.noir-ref || 'master' }}

- name: Setup for Apple Silicon
if: matrix.target == 'aarch64-apple-darwin'
run: |
sudo xcode-select -s /Applications/Xcode_13.2.1.app/Contents/Developer/
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
echo "SDKROOT=$(xcrun -sdk macosx$(sw_vers -productVersion) --show-sdk-path)" >> $GITHUB_ENV
echo "MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx$(sw_vers -productVersion) --show-sdk-platform-version)" >> $GITHUB_ENV

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build environment and Compile
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo build --package nargo_cli --release --target ${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-${{ matrix.target }}.tar.gz
kevaundray marked this conversation as resolved.
Show resolved Hide resolved

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./dist/*
retention-days: 3

- name: Test built artifact
if: matrix.target == 'x86_64-apple-darwin'
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/

cd release-tests
yarn install
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
yarn test

# - name: Upload binaries to Noir Repo
# uses: svenstaro/upload-release-action@v2
# if: ${{ inputs.publish || github.event_name == 'schedule' }}
# with:
# repo_name: noir-lang/noir
# repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
# file: ./nargo-${{ matrix.target }}.tar.gz
# asset_name: nargo-${{ matrix.target }}.tar.gz
# overwrite: true
# tag: ${{ inputs.noir-ref || 'nightly' }} # This will fail if noir-ref is not a tag (e.g. testing a branch)

# # If we're performing a nightly build and it fails acceptance tests then raise an issue.
# - name: Alert on nightly build failure
# uses: JasonEtco/create-an-issue@v2
# if: ${{ failure() && github.event_name == 'schedule' }}
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# WORKFLOW_NAME: ${{ github.workflow }}
# WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# with:
# update_existing: true
# filename: .github/NIGHTLY_BUILD_FAILURE.md

build-linux:
needs: [build-barretenberg]
runs-on: ubuntu-22.04
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
fail-fast: false
matrix:
target:
[
x86_64-unknown-linux-gnu,
x86_64-unknown-linux-musl,
aarch64-unknown-linux-gnu,
aarch64-unknown-linux-musl,
]

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Checkout Noir repo
uses: actions/checkout@v3
with:
ref: ${{ inputs.noir-ref || 'master' }}

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build Nargo
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo install cross --force --git https://github.com/cross-rs/cross
cross build --package nargo_cli --release --target=${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-${{ matrix.target }}.tar.gz

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./dist/*
retention-days: 3

- name: Test built artifact
if: startsWith(matrix.target, 'x86_64-unknown-linux')
run: |
cp ./target/${{ matrix.target }}/release/nargo ~/.cargo/bin/

cd release-tests
yarn install
yarn test

# - name: Upload binaries to Noir Repo
# uses: svenstaro/upload-release-action@v2
# if: ${{ inputs.publish || github.event_name == 'schedule' }}
# with:
# repo_name: noir-lang/noir
# repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
# file: ./nargo-${{ matrix.target }}.tar.gz
# asset_name: nargo-${{ matrix.target }}.tar.gz
# overwrite: true
# tag: ${{ inputs.noir-ref || 'nightly' }} # This will fail if noir-ref is not a tag (e.g. testing a branch)

# # If we're performing a nightly build and it fails acceptance tests then raise an issue.
# - name: Alert on nightly build failure
# uses: JasonEtco/create-an-issue@v2
# if: ${{ failure() && github.event_name == 'schedule' }}
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# WORKFLOW_NAME: ${{ github.workflow }}
# WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# with:
# update_existing: true
# filename: .github/NIGHTLY_BUILD_FAILURE.md

build-windows:
needs: [build-barretenberg]
runs-on: windows-2022
env:
CROSS_CONFIG: ${{ github.workspace }}/.github/Cross.toml
strategy:
matrix:
target: [x86_64-pc-windows-msvc]

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Checkout Noir repo
uses: actions/checkout@v3
with:
ref: ${{ inputs.noir-ref || 'master' }}

- uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: libbarretenberg-wasm32
path: ${{ github.workspace }}/libbarretenberg-wasm32

- name: Setup toolchain
uses: dtolnay/rust-toolchain@1.66.0
with:
targets: ${{ matrix.target }}

- name: Build environment and Compile
env:
BARRETENBERG_BIN_DIR: ${{ github.workspace }}/libbarretenberg-wasm32/bin
run: |
cargo build --package nargo_cli --release --target ${{ matrix.target }} --no-default-features --features plonk_bn254_wasm

- name: Package artifacts
run: |
mkdir dist
cp ./target/${{ matrix.target }}/release/nargo.exe ./dist/nargo.exe
7z a -tzip nargo-${{ matrix.target }}.zip ./dist/*

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo-${{ matrix.target }}
path: ./dist/*
retention-days: 3

- name: Test built artifact
shell: powershell
run: |
cp ./target/${{ matrix.target }}/release/nargo.exe ~/.cargo/bin/

cd release-tests
yarn install
yarn test

# - name: Upload binaries to Noir Repo
# uses: svenstaro/upload-release-action@v2
# if: ${{ inputs.publish || github.event_name == 'schedule' }}
# with:
# repo_name: noir-lang/noir
# repo_token: ${{ secrets.NOIR_REPO_TOKEN }}
# file: ./nargo-${{ matrix.target }}.zip
# asset_name: nargo-${{ matrix.target }}.zip
# overwrite: true
# tag: ${{ inputs.noir-ref || 'nightly' }} # This will fail if noir-ref is not a tag (e.g. testing a branch)

# # If we're performing a nightly build and it fails acceptance tests then raise an issue.
# - name: Alert on nightly build failure
# uses: JasonEtco/create-an-issue@v2
# if: ${{ failure() && github.event_name == 'schedule' }}
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# WORKFLOW_NAME: ${{ github.workflow }}
# WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# with:
# update_existing: true
# filename: .github/NIGHTLY_BUILD_FAILURE.md
15 changes: 15 additions & 0 deletions release-tests/package.json
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "@noir-lang/release-tests",
"version": "0.0.0",
"main": "index.js",
"license": "MIT",
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
"private": true,
"type": "module",
"scripts": {
"test": "node ./node_modules/uvu/bin.js test"
},
"devDependencies": {
"uvu": "0.5.6",
"zx": "7.1.1"
}
}
Loading
Loading