Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul the NuGet packaging infrastructure and the way the Core binaries get acquired. #146

Merged
merged 25 commits into from
Dec 1, 2022
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0c36a4c
Start writing an MSBuild script that creates NuGet packages for the c…
teo-tsirpanis Nov 14, 2022
1d7d3b0
Support creating the native packages.
teo-tsirpanis Nov 14, 2022
61b5178
Support creating the metapackage.
teo-tsirpanis Nov 15, 2022
2a06a80
(REMOVE BEFORE MERGE) Add a temporary MyGet feed for the TileDB nativ…
teo-tsirpanis Nov 15, 2022
b72b8dc
Use the TileDB Embedded page for the project URL in the native packages.
teo-tsirpanis Nov 15, 2022
cad8b05
Use the `TileDB.Native` NuGet package and remove the nuspec of `TileD…
teo-tsirpanis Nov 15, 2022
74a1520
Set `UseCurrentRuntimeIdentifier` in the executable projects.
teo-tsirpanis Nov 15, 2022
2ec7fe9
Restore the template packages.
teo-tsirpanis Nov 15, 2022
287a4ac
Remove the ValueSeparator hack.
teo-tsirpanis Nov 15, 2022
0e4ca34
Remove the "Install/Download tiledb" steps from CI.
teo-tsirpanis Nov 15, 2022
c14741d
Remove all CI jobs except of Run-Tests.
teo-tsirpanis Nov 16, 2022
3c33ffe
Remove RollForward from TileDB.CSharp.
teo-tsirpanis Nov 16, 2022
e655b26
Update package metadata and move them to a dedicated file.
teo-tsirpanis Nov 17, 2022
d0764a6
Add support for generating development editions of the native packages.
teo-tsirpanis Nov 17, 2022
4d85390
Enable Central Package Management with package source mapping.
teo-tsirpanis Nov 17, 2022
f848e09
Add release notes in the packages.
teo-tsirpanis Nov 17, 2022
8869749
Refactor the nightly build workflow to use native development NuGet p…
teo-tsirpanis Nov 17, 2022
fffdd3c
Update the repository README and add a package README.
teo-tsirpanis Nov 17, 2022
4f3601f
Require `Version` and `VersionTag` to be specified.
teo-tsirpanis Nov 17, 2022
f787a3c
Add a README describing the native NuGet package generator.
teo-tsirpanis Nov 17, 2022
abf0603
Demand Core version 2.12.x.
teo-tsirpanis Nov 23, 2022
e456ae3
Warn when building an RID-agnostic executable that uses `TileDB.CSharp`.
teo-tsirpanis Nov 24, 2022
ec1ddab
Remove the RID-specific build instructions in the metapackage's descr…
teo-tsirpanis Nov 24, 2022
24b425d
Remove the MyGet feed; we are now using the official TileDB binaries …
teo-tsirpanis Dec 1, 2022
dd9213a
Use MSBuild's custom item separator feature instead of a `string.Repl…
teo-tsirpanis Dec 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ indent_style = space
indent_size = 4
insert_final_newline = true

[*.{xml,*proj,props,targets,yml}]
[*.{xml,*proj,props,targets,yml,nuspec}]
indent_size = 2

[*.cs]
Expand Down
18 changes: 0 additions & 18 deletions .github/scripts/download_tiledb.sh

This file was deleted.

4 changes: 0 additions & 4 deletions .github/scripts/install_tiledb.sh

This file was deleted.

129 changes: 87 additions & 42 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -1,90 +1,135 @@
name: Nightly-Build

env:
TILEDB_NIGHTLY_BUILD: 1

on:
schedule:
- cron: "15 2 * * *"
workflow_dispatch:

jobs:
Build:
Build-Native:
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
include:
- os: ubuntu-22.04
HOST: linux
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
- BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
- os: ubuntu-latest
HOST: linux
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
platform: linux-x86_64
- os: macos-latest
HOST: osx
BOOTSTRAP: ../bootstrap --enable-s3 --enable-serialization
platform: macos-x86_64
- os: windows-latest
HOST: win
platform: windows-x86_64
BOOTSTRAP: ../bootstrap.ps1 -EnableS3 -EnableSerialization
tag: [release-2.12, dev]
dotnet: ['net5.0', 'net6.0']
runs-on: ${{ matrix.os }}

steps:
- name: Checkout TileDB-CSharp
uses: actions/checkout@v3
with:
path: tiledb-csharp

- name: Checkout TileDB
uses: actions/checkout@v3
with:
repository: TileDB-Inc/TileDB
ref: ${{ matrix.tag }}
path: tiledb

- name: Build TileDB
run: |
mkdir -p build
cd build
${{ matrix.BOOTSTRAP }}
cmake --build . --config Release
cmake --build . --config Release --target install-tiledb
cd ../
mkdir -p artifacts/${{ matrix.platform }}
mv dist/* artifacts/${{ matrix.platform }}

# We combine the binaries of the many platforms per tag by uploading them to the same artifact.
- name: Upload native artifacts
uses: actions/upload-artifact@v3
with:
name: tiledb-native-${{ matrix.tag }}
path: artifacts/

Pack-NuGet:
needs: Build-Native
strategy:
fail-fast: false
matrix:
tag: [release-2.12, dev]
runs-on: ubuntu-latest
steps:
- name: Checkout TileDB-CSharp
uses: actions/checkout@v3

# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
- name: Remove existing .NET versions
shell: bash
run: |
rm -rf $DOTNET_ROOT
run: rm -rf $DOTNET_ROOT

- name: Set up .NET SDK from global.json
uses: actions/setup-dotnet@v3
with:
global-json-file: tiledb-csharp/global.json

- name: Display dotnet versions
run: dotnet --info

- name: Build TileDB
run: |
mkdir -p tiledb/build
cd tiledb/build
${{ matrix.BOOTSTRAP }}
cmake --build . --config Release
cmake --build . --config Release --target install-tiledb
cd ../../
cp -r tiledb/dist/lib/* tiledb-csharp/sources/TileDB.CSharp/runtimes/${{ matrix.HOST }}-x64/native/
- name: Download native artifacts
uses: actions/download-artifact@v3
with:
name: tiledb-native-${{ matrix.tag }}
path: scripts/nuget/temp

- name: Copy Windows tiledb.dll
if: matrix.HOST == 'win'
run: |
cp tiledb/dist/bin/tiledb.dll tiledb-csharp/sources/TileDB.CSharp/runtimes/${{ matrix.HOST }}-x64/native/
- name: Build native NuGet packages
shell: bash
run: dotnet pack ./scripts/nuget/GenerateNuGetPackages.proj -p:DevelopmentBuild=true -p:VersionTag=${{ matrix.tag }}

- name: Upload native NuGet packages
uses: actions/upload-artifact@v3
with:
name: tiledb-nuget-${{ matrix.tag }}
path: scripts/nuget/packages/

Test-NuGet:
needs: Pack-NuGet
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, ubuntu-latest, macos-latest, windows-latest]
tag: [release-2.12, dev]
dotnet: ['net5.0', 'net6.0']
runs-on: ${{ matrix.os }}
steps:
- name: Checkout TileDB-CSharp
uses: actions/checkout@v3

# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
- name: Remove existing .NET versions
shell: bash
run: rm -rf $DOTNET_ROOT

- name: Set up .NET SDK from global.json
uses: actions/setup-dotnet@v3

- name: Display dotnet versions
run: dotnet --info

- name: Download native NuGet packages
uses: actions/download-artifact@v3
with:
name: tiledb-nuget-${{ matrix.tag }}
path: packages

- name: Build TileDB-CSharp
run: |
dotnet build tiledb-csharp/sources/TileDB.CSharp/TileDB.CSharp.csproj /p:Platform=x64 -c Release
run: dotnet build sources/TileDB.CSharp/TileDB.CSharp.csproj -c Release

- name: Test TileDB-CSharp
run: |
dotnet test tiledb-csharp/tests/TileDB.CSharp.Test/TileDB.CSharp.Test.csproj -c Release -f ${{ matrix.dotnet }}
run: dotnet test tests/TileDB.CSharp.Test/TileDB.CSharp.Test.csproj -c Release -f ${{ matrix.dotnet }}

- name: Run examples
shell: bash
run: |
find tiledb-csharp/examples/ -name *.csproj -execdir dotnet run \;
run: find examples/ -name *.csproj | xargs -I{} dotnet run --project {} -f ${{ matrix.dotnet }}

Create-Issue:
needs: Build
needs: Test-NuGet
if: failure()
runs-on: ubuntu-latest
steps:
Expand Down
102 changes: 2 additions & 100 deletions .github/workflows/tiledb-csharp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ name: TileDB-CSharp

on:
push:
tags: [ '*' ]
branches: [ main ]
pull_request:
branches: [ main ]
Expand All @@ -11,6 +10,7 @@ on:
jobs:
Run-Tests:
strategy:
fail-fast: false
matrix:
# Will be checking following versions
dotnet: ['net5.0', 'net6.0']
Expand All @@ -21,11 +21,6 @@ jobs:
# Checks out repository
- uses: actions/checkout@v3

# Install tiledb
- name: Install tiledb
shell: bash
run: ./.github/scripts/install_tiledb.sh

- name: Remove existing .NET versions
shell: bash
run: |
Expand All @@ -40,7 +35,7 @@ jobs:
# DotNet build
- name: Dotnet build for TileDB.CSharp
run: |
dotnet build /p:Platform=x64 -c Release sources/TileDB.CSharp
dotnet build -c Release sources/TileDB.CSharp

# DotNet test
- name: Test TileDB.CSharp
Expand All @@ -51,96 +46,3 @@ jobs:
shell: bash
run: |
find examples/ -name *.csproj | xargs -I{} dotnet run --project {} -f ${{ matrix.dotnet }}

Stage-Release-Candidate:
needs: Run-Tests
runs-on: ubuntu-latest
steps:
# Checks out repository
- uses: actions/checkout@v3

- name: Set up .NET SDK from global.json
uses: actions/setup-dotnet@v3

- name: Display dotnet versions
run: dotnet --info

# Download tiledb
- name: Download tiledb
run: ./.github/scripts/download_tiledb.sh

# DotNet build
- name: Dotnet build for TileDB.CSharp
run: |
dotnet build /p:Platform=x64 -c Release sources/TileDB.CSharp

# DotNet pack
- name: Dotnet pack for TileDB.CSharp
run: |
dotnet pack ./sources/TileDB.CSharp/TileDB.CSharp.csproj -p:Version=0.0.0-local -c Release

- name: Archive nuget artifact
uses: actions/upload-artifact@v3
with:
name: TileDB NuGet Package
path: sources/TileDB.CSharp/lib/TileDB.CSharp.*.nupkg

Test-NuGet-Release:
needs: Stage-Release-Candidate
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
dotnet: ['net5.0', 'net6.0']
runs-on: ${{ matrix.os }}
steps:
- name: Checkout TileDB-CSharp repository
uses: actions/checkout@v3

- name: Download TileDB.CSharp NuGet artifact
uses: actions/download-artifact@v3
with:
name: TileDB NuGet Package
path: packages

# GitHub runners come with several versions of .NET preinstalled; Remove them to target version
- name: Remove existing .NET versions
shell: bash
run: |
rm -rf $DOTNET_ROOT

- name: Set up .NET SDK from global.json
uses: actions/setup-dotnet@v3

- name: Display dotnet versions
run: dotnet --info

- name: Setup NuGet
uses: nuget/setup-nuget@v1

# Run tests using NuGet release candidate
- name: Test TileDB.CSharp
run: |
dotnet test -c Release tests/TileDB.CSharp.Test.NuGet -f ${{ matrix.dotnet }}

Release:
# Only run this job if a tag was provided
if: startsWith(github.ref, 'refs/tags/')
needs: Test-NuGet-Release
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v3
with:
name: TileDB NuGet Package

- name: Create Release
id: create_release
uses: softprops/action-gh-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
files: ./*.nupkg
tag_name: ${{ github.ref_name }}
name: ${{ github.ref_name }}
body: ${{ steps.github_release.outputs.changelog }}
draft: false
prerelease: false
6 changes: 4 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,6 @@

TileDB-CSharp.sln.DotSettings.user

build

TileDBConfig.cmake
TileDbTargets*.cmake
tiledb.pc
Expand All @@ -54,3 +52,7 @@ tiledb.pc

.DS_Store
include
scripts/nuget/temp
*.binlog
*.nupkg
packages
3 changes: 0 additions & 3 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,4 @@
<PropertyGroup>
<LanguageVersion>latest</LanguageVersion>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="SonarAnalyzer.CSharp" Version="8.48.0.56517" PrivateAssets="all" />
</ItemGroup>
</Project>
Loading