-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Overhaul the NuGet packaging infrastructure and the way the Core binaries get acquired. #146
Conversation
…ore. For now it downloads the artifacts from GitHub Releases.
And add a property to them that seems to save time.
And disable fail-fast in the Run-Tests action matrix.
Now that we don't use a nuspec to pack the C# library there are little reasons to specifically test it with a packed NuGet package. The Release job also went away; it would upload the 0.0.0-local package. We need a dedicated release workflow.
It does not apply to libraries.
I had originally removed the nuspec files from the native packages but changed my mind for aesthetic reasons since they generate an empty .NET Standard 2.0 dependency group.
They have SemVer 2.0 versions that identify the branch.
And support changing the native package under a different name in development builds.
…ackages. The core is built in separate jobs and not once for each .NET version.
Make the version tag optional for development builds. And fail if no native binaries were found in development builds.
Its important we always tie C# releases to specific core version. This is critical for support and understanding when a user is on C# x.y.z what core version that ties too. We keep a version mapping and compatibility for our APIs to help facilitate this. Its also important to tie to specific version for deprecation cycles. |
OK, we can easily make |
I added a warning for those who try to build an RID-agnostic executable that depends on Adding the warning to The .NET SDK is planning to enable RID-specific builds by default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is really nice overall, 👍 to merge. But as a follow-up, I'd like to figure out how we can make the dev experience against a local core/libtiledb build more straightforward. For reference, with tiledb-py we can pass a --tiledb
argument to setup.py, pointing to the core dist/
folder, and then I can just make changes and update the core build iteratively to test fixes (only rebuilding py bindings if I make a change on the python side). I'd like to aim for a similar workflow here in the future, if at all possible w/in the constraints of dotnet.
…iption. This package is not intended to be directly referenced by the average developer, at least not without `TileDB.CSharp`.
Great! Before I merge it we have to push the native packages to NuGet. Check the |
There is room for improvement but until then, a trick that has helped me in the past is to replace the library in |
I removed the MyGet feed. CI is green and nightly builds succeed. Merging. |
When a user installs the
TileDB.CSharp
package from NuGet, they get theTileDB.CSharp.dll
managed assembly, but also the native Core binaries for Windows, macOS and Linux on x64. This existing approach has many disadvantages:This PR fixes all of the above problems. The Core gets distributed via separate NuGet packages for each supported RID, called
TileDB.Native.runtime.<rid>
, and one metapackage that tells the .NET SDK which of the above to use, calledTileDB.Native
. The entire process is automated and all necessary packages for a published Core GitHub Release can be built with one command.Advantages
The advantages are enormous:
TileDB.CSharp
package contains just the managed code and depends onTileDB.Native
for the rest, significantly shrinking itself to just 54KB.TileDB.CSharp
you will download none!TileDB.CSharp
package is also no more built with a.nuspec
file; everything is controlled using regular project properties.Drawbacks
Not necessarily drawbacks, but some things will change. They have been appropriately documented.
Users will need to explicitly opt into RID-specific builds. Just
dotnet run
ning a C# app that uses TileDB will fail with aDllNotFoundException
. They will need to add a project property to their executable projects that enables .NET to download the RID-specific assets. See theUseCurrentRuntimeIdentifier
property I added to the test and example projects.Using custom Core builds got a little more complicated. The expected way to do it is to download the NuGet packages from a nightly build's artifacts and follow the instructions in
Directory.Packages.props
. If you want to test with your own Core build, it will be slightly more complicated, but there are instructions.I cut down the
tiledb-csharp
pipeline, in particular theStage-Release-Candidate
,Test-NuGet-Release
andRelease
jobs. The former two are not needed; the construction of theTileDB.CSharp
NuGet package is handled entirely by the .NET SDK; there is no benefit in testing it ourselves, and theRelease
job will need to be rewritten into its own workflow, which I will do in a future PR.Before merging this
To unblock myself from working on this, I temporarily uploaded the native packages to a personal MyGet feed of mine. Assuming we want these changes, we should upload the packages to the official nuget.org feed before merging this PR.
To create the packages checkout the PR's branch and run
It will produce some
nupkg
files to thescripts\nuget\packages
folder; upload them to NuGet.