Where to put Native libraries inside a Project and how to reference them?

[morguldir]

I want a way to have native libraries added to NATIVE_DLL_SEARCH_DIRECTORIES inside of a project, so that if something imports the project as a Project Reference, they will get added to the search directories. I already know how to do this with a nuget package, where i put them inside runtimes/{rid}/native. It is a c# library, but i need the native libraries to work for the tests in the same location as well.

I could move the native libraries out of the repo into a seperate nuget package, but i want some kind of confirmation that i should do it that way. I also think there should be some documentation for this.

[jeffschwMSFT]

@morguldir I agree moving these into a nuget package will provide the semantics that I feel you are looking for. I am adding a few folks that would know if there is another way.

cc @jkoritzinsky @sdmaclea

[jkoritzinsky]

Moving the native libraries into their own NuGet package may be the easiest way of doing what you want. There are some other options though:

One option (which would also enable you to easily add the native libraries to a NuGet package), would be to add the following (or something similar) to your .csproj.

<Content Include="path/to/my/native/lib.dll">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    <Pack>false</Pack>
</Content>

If you also want to pack it in the same NuGet package as the csproj (i.e. you are able to have all of the required native libraries available on the same machine at build time), you could do something as follows for each of the native libraries:

<Content Include="path/to/my/native/lib.dll">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    <Pack>true</Pack>
    <PackagePath>runtimes/win-x64/native/lib.dll</PackagePath>
</Content>

[morguldir]

A few problems with the CopyToOutputDirectory one, it won't work for linux, doesn't look like it works for different architectures and it uses more space than needed. There was talk about putting the files into *.deps.json, but it looks like that is only for nuget packages?

[sdmaclea]

There does not seem to be a reason @jkoritzinsky's approach wouldn't work for multiple architecture and for Linux. You would of course need to include all the native os/architecture variants. Windows, Linux (glibc), Linux (musl) Mac, x64, x86, arm, arm64... Something like 16 native library variants.

If package size is a problem, separate nuget package would allow downloading the correct variant for you platform.

[morguldir]

Right, you could just use conditions for architecture, my bad. However a big problem is that Linux does not look for native libraries in the current directory, package size isn't that much of a problem, but copying it to every output seems inefficient, you already know where the file is.

[sdmaclea]

However a big problem is that Linux does not look for native libraries in the current directory

@swaroop-sridhar Has been actively working on native library features and may be able to comment.

I suspect you are right for the typical dotnet myapp.dll case. If the app is published as a self contained app, it will at least search near the executable which will be in the same directory as the managed code. dotnet publish --self-contained.

I think it will look beside the executable. So if the app is deployed as a standalone app, it should, as long as the installer places the native library in an appropriate place. It still will not search in subdirectories. So you will likely need an installer to move the library to the correct location.

@swaroop-sridhar Will the dll map effort reduce the pain here?

[sdmaclea]

@vitek-karas FYI

[swaroop-sridhar]

If you want to load a native library from a non-standard location (ex: other than the nuget cache, output or publish directories), the upcoming nativelibrary APIs to load from any location will help: https://github.com/dotnet/corefx/issues/32015

The intention of this API is to deal with customization of dynamic native library loads. If a native library reference comes through a static project reference in the same code repo, I'd recommend generating it as an output.

Have you tried adding a DllImportSearchPath attribute to the referencing assembly to search AssemblyDirectory or ApplicationDirectory?

What is the Linux specific issue here? Do you find NATIVE_DLL_SEARCH_DIRECTORIES different in Linux and Windows?

[vitek-karas]

I'm not convinced that the dll map features are the right solution here. They all require code in the app - effectively one would have to add knowledge of the build system into the app a little bit. That's exact what .deps.json is trying to avoid.
The right thing here would be a way to declare the native library as part of the app in the project file in such a way that the SDK would put it into the .deps.json. As noted this can be done through NuGet packages, but I'm not aware of any way to do this within the app itself, which seems like a hole in the design.

@livarcocc - is this something the SDK might be able to do?

[morguldir]

Have you tried adding a DllImportSearchPath attribute to the referencing assembly to search AssemblyDirectory or ApplicationDirectory?

What is the Linux specific issue here? Do you find NATIVE_DLL_SEARCH_DIRECTORIES different in Linux and Windows?

Looks like DllImportSearchPath only works in the code that has the DllImport in them, we also have DllImports in external libraries. The linux issue isn't with coreclr, linux doesn't look inside the current directory for native libs unlike other platforms.

I'm not convinced that the dll map features are the right solution here. They all require code in the app - effectively one would have to add knowledge of the build system into the app a little bit. That's exact what .deps.json is trying to avoid.
The right thing here would be a way to declare the native library as part of the app in the project file in such a way that the SDK would put it into the .deps.json. As noted this can be done through NuGet packages, but I'm not aware of any way to do this within the app itself, which seems like a hole in the design.

@livarcocc - is this something the SDK might be able to do?

From what i have seen it doesn't look like .deps.json has a field that isn't for nuget paths.

[vitek-karas]

I'm pretty sure .deps.json can express this. Managed assemblies are referenced like:

{
  "runtime": {
    "MyManagedApp.dll": {}
  }
}

Native libraries can be referenced on the same level like this:

{
  "native": {
    "linux-x64/mynative.so": {}
  }
}

Typically the native libraries are in the RID specific section, so for example in ".NETCoreApp,Version=v2.1/ubuntu.14.04-x64", so that on a any given platform the right native library is picked.

What I don't know is if there's a way in the project file to add a native library in such a way that the SDK will generate a record like above into the .deps.json.

[sdmaclea]

The linux issue isn't with coreclr, linux doesn't look inside the current directory for native libs unlike other platforms

This is the default behavior. There are linker setting on Linux which allow libraries and apps to influence the search path for their dependent libraries. If you search for rpath you will find the relevant information.

[morguldir]

I'm pretty sure .deps.json can express this. Managed assemblies are referenced like:

{
  "runtime": {
    "MyManagedApp.dll": {}
  }
}

Native libraries can be referenced on the same level like this:

{
  "native": {
    "linux-x64/mynative.so": {}
  }
}

Typically the native libraries are in the RID specific section, so for example in ".NETCoreApp,Version=v2.1/ubuntu.14.04-x64", so that on a any given platform the right native library is picked.

What I don't know is if there's a way in the project file to add a native library in such a way that the SDK will generate a record like above into the .deps.json.

Oh okay, could you make that work without copying the file to the output directory?

[vitek-karas]

In theory the paths in the .deps.json can point anywhere, but I don't think the tooling will do that. So if you manually created/modified the .deps.json then it will probably work (never tried this with absolute paths). That said modifying the .deps.json is a tedious process and build will blindly overwrite the file. The file was meant to be machine generated (and consumed by machines), so not exactly human friendly.

[morguldir]

Just want to know how it will work when it is implemented, if you use a nuget package now, you can just copy the output and remove the original directory and the output will work, so you won't have to make a self-contained app. Would having a nuget directory for native libraries be possible?

[vitek-karas]

I don't know myself. @livarcocc or somebody from his team should be able to answer that.

[morguldir]

@sdmaclea You mentioned being able to download only the native libraries that you needed for your platform, is this possible with one nuget package, or do you need it to depend on other nuget packages based on the platform?

[jkotas]

You need multiple nuget packages for this. Here is an example how it is done for .NET Core package itself (this is used for self-contained apps):

https://www.nuget.org/packages/Microsoft.NETCore.App/
https://www.nuget.org/packages/runtime.win-x64.Microsoft.NETCore.App/
https://www.nuget.org/packages/runtime.linux-x64.Microsoft.NETCore.App/

[jkoritzinsky]

@jkotas do we have any documentation on constructing packages like that (and having them auto-resolved by NuGet based on RID)? I've spent a solid amount of time in the past trying to find documentation on how to do that and have come up empty.

[jkotas]

@ericstj Do we have documentation for how to create NuGet packages with native assets?

[ericstj]

Here are "official" docs: https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks.

Note that the way CoreCLR and CoreFx do this using runtime.json isn't something that NuGet recommends generally. runtime.json was created specially for the framework itself, never officially documented and its something we are planning on moving away from in the 3.0 release with runtime packs resolved by the SDK instead of NuGet.

I recommend that you take this discussion over to https://github.com/nuget/home. Here's a related issue: NuGet/Home#1660.

[jeffschwMSFT]

Closing out this question. Please let us know if you have any additional questions.