VC Project not support nuget PackageReference style

[mingkuang-Chuyu]

Maintainer edit:

Please refer to https://developercommunity.visualstudio.com/t/use-packagereference-in-vcxproj/351636

VC Project not support nuget PackageReference style

NuGet product used VS UI:
NuGet version 5.3.1.6268
VS version 16.3.10
OS version Windows 7 SP1 x64

As we all know, packages.xml easily leads to wasted space and a large number of IO copy operations. So I hope that nuget can support VC++ to support PackageReference style.

After research, vs itself does not provide PackageReference for VC++, but nuget can directly call MSBuild API to modify "ItemGroup/PackageReference" node to implement VC++ PackageReference style.

The modified work looks great and has fewer changes.

I also provided PR, and hope to get everyone's approval. After all, it is almost 2020, Nuget still does not support PackageReference for VC++.

Thanks

mingkuang

My RP(add C++ PackageReference support)

NuGet.Client

NuGet/NuGet.Client#3145

NuGet.BuildTasks

dotnet/NuGet.BuildTasks#67

[xspeed1989]

nice ！

[rrelyea]

@mingkuang-Chuyu - Are you suggesting, that if we call msbuild APIs from NuGet code, we can fix this for nuget command line scenarios and VS scenarios too?
Did you test your PR in all those scenarios?
This issue needs more details than you have provided so far.

[mingkuang-Chuyu]

@rrelyea Yes, I have tested nuget in all scenarios I know.

nuget.exe restore

result : is OK

nuget restore "I:\Visual Studio 2013\Projects\PluginsMgr\Cons
oleApplication1\ConsoleApplication1.vcxproj"

Log

MSBuild auto-detection: using msbuild version '16.3.2.50909' from 'C:\Program Fi
les (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\bin'.
Restoring packages for I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplicat
ion1\ConsoleApplication1.vcxproj...
Committing restore...
Generating MSBuild file I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplica
tion1\obj\ConsoleApplication1.vcxproj.nuget.g.props.
Generating MSBuild file I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplica
tion1\obj\ConsoleApplication1.vcxproj.nuget.g.targets.
Writing assets file to disk. Path: I:\Visual Studio 2013\Projects\PluginsMgr\Con
soleApplication1\obj\project.assets.json
Restore completed in 606.35 ms for I:\Visual Studio 2013\Projects\PluginsMgr\Con
soleApplication1\ConsoleApplication1.vcxproj.

NuGet Config files used:
    C:\Users\xupengjie\AppData\Roaming\NuGet\NuGet.Config
    C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

Feeds used:
    C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
    I:\NuGet\PackageTmp
    I:\NuGet\Soucre
    https://api.nuget.org/v3/index.json

msbuild.exe restore

result : is OK
msbuild -restore -p:Configuration=Release;Platform=Win32 "I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj"

Log

用于 .NET Framework 的 Microsoft (R) 生成引擎版本 16.3.2+e481bbf88
版权所有(C) Microsoft Corporation。保留所有权利。

生成启动时间为 2019/12/6 10:47:44。
项目“I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj”在节点 1 上(Restore 个目标)。
Restore:
  正在还原 I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj 的包...
  正在提交还原...
  正在生成 MSBuild 文件 I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\obj\ConsoleApplication1.vcxproj.nuget.g.props。
  正在生成 MSBuild 文件 I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\obj\ConsoleApplication1.vcxproj.nuget.g.targets。
  将资产文件写入磁盘。路径: I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\obj\project.assets.json
  I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj 的还原在 537.21 ms 内完成。

  使用的 NuGet 配置文件:
      C:\Users\xupengjie\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  使用的源:
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
      I:\NuGet\PackageTmp
      I:\NuGet\Soucre
已完成生成项目“I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj”(Restore 个目标)的操作。

[rrelyea]

And in visual studio? What is the behavior? (Without running a Commandline restore ever on that project)

[mingkuang-Chuyu]

nuget.exe, msbuild.exe and VS NuGet UI, all works fine.

[mingkuang-Chuyu]

in VS UI Build
Log

Restoring packages for I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\ConsoleApplication1.vcxproj...
Installing VC-LTL-v142 4.0.2.14.
Committing restore...
Generating MSBuild file I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\obj\ConsoleApplication1.vcxproj.nuget.g.props.
Writing assets file to disk. Path: I:\Visual Studio 2013\Projects\PluginsMgr\ConsoleApplication1\obj\project.assets.json
Successfully installed 'VC-LTL-v142 4.0.2.14' to ConsoleApplication1
Executing nuget actions took 540.35 ms
Time Elapsed: 00:00:28.1483594
========== Finished ==========

[mingkuang-Chuyu]

And Build log : )

1>------ 已启动生成: 项目: ConsoleApplication1, 配置: Release Win32 ------
1>
1>正在生成代码
1>0 of 147 functions ( 0.0%) were compiled, the rest were copied from previous compilation.
1>  0 functions were new in current compilation
1>  0 functions had inline decision re-evaluated but remain unchanged
1>已完成代码的生成
1>ConsoleApplication1.vcxproj -> I:\Visual Studio 2013\Projects\PluginsMgr\Release\ConsoleApplication1.exe
1>已完成生成项目“ConsoleApplication1.vcxproj”的操作。

[karann-msft]

What version of Visual Studio did you test this with? Logs suggest 2013.

[mingkuang-Chuyu]

@karann-msft Hi, VS2019 version is 16.3.10.

: )

[mingkuang-Chuyu]

But VS2017 also works.

[zivkan]

For .NET projects, PackageReference works by using MSBuild to find the full list of PackageReferences not only for the current project, but also transitive PackageReferences from project-to-project references (although c++ projects can use ProjectReference, it's unclear to me what this does other than specify build order). It restores the packages (and their dependencies) to the global packages folder, then it writes a project.assets.json file which specifies all the compile and runtime assets, consumed by the project system, but also writes a .g.props and .g.targets so that the props/targets files from packages can be discovered, also consumed by the project system.

So, when you propose that NuGet uses MSBuild APIs to modify ItemGroup/PackageReference, how exactly will it be modified, what is going to be put into the project? Is it adding package assets, or props/targets files, or both? Is it saving the changes to the vcproj to disk, like packages.config does, or just modifying it in memory? Without project system support, how do packages that bring props/targets file have those props/targets files evaluated, given the project has already been evaluated before the packages were restored?

In .NET projects, packages.config allows files from the package to be added to the project using the content directory. With PackageReference it works differently, and uses the contentFiles directory (via the assets file I mentioned earlier). Will it be the same for c++ projects? how about source file transformations?

In .NET projects, PackageReference separates the concept of compile-time assets from runtime assets. Does the same distinction make sense for native/c++ projects? In .NET, the package's runtime folder uses a RID to use different assets depending on the runtime (linux vs mac vs windows, even ubutu vs redhat, 64-bit vs 32 bit vs arm). RIDs are concepts that cannot be evaluated at NuGet restore time, or package add time, as a project may multi-target. How does this fit into c++ projects, particularly without c++ project system support?

Those are all the features of PackageReference that I can think of, but there may be other things. I really do appriciate anyone spending their effort contributing, but once the NuGet team ships something, we have to support it, so it needs to be well designed.

[mingkuang-Chuyu]

@zivkan ，
There may be some problems with contentfiles support. There are no problems with other aspects. It has been tested.

.g.props/ .g.targets is ok(nuget,msbuild, vs nuget ui).
restore is ok(nuget,msbuild, vs nuget ui).
RID is ok (nuget,msbuild, vs nuget ui).

contentFiles，I'm not sure. I'll give you an answer tomorrow.

I have provided logs to prove that they work properly.

Thanks for your support.

[mingkuang-Chuyu]

@zivkan contentFiles is not work. Do you know where the "project.assets.json" generated code is?

Thanks

[mingkuang-Chuyu]

@zivkan @rrelyea @rrelyea I have added contentFiles support. Now that all functions are fully implemented, please help to see if they can be merged?

[mingkuang-Chuyu]

contentFiles is ok. This is the test configuration

<package> 
  <metadata minClientVersion="3.3.0"> 
    <id>VC.contentFiles.Test</id> 
    <version>1.0.4</version> 
    <authors>nuget</authors> <!-- The NuGet team authored this package --> 
    <owners>nuget</owners> <!-- The NuGet team owns this package --> 
    <requireLicenseAcceptance>false</requireLicenseAcceptance> 
    <description>A content v2 example package.</description> 
    <tags>contentv2 contentFiles</tags> 
    <!-- Build actions for items in the contentFiles folder --> 
    <contentFiles>
      <!--C++ Srorce File-->
      <files include="cpp/**/*.cpp" buildAction="ClCompile" /> 
       <!--C Srorce File-->
      <files include="cpp/**/*.c" buildAction="ClCompile" /> 
       <!--asm Srorce File-->
      <files include="cpp/**/*.asm" buildAction="MASM" /> 
       <!--Resource File-->
      <files include="cpp/**/*.rc" buildAction="ResourceCompile" /> 
      <!--Interface Definition Language File-->
      <files include="cpp/**/*.ild" buildAction="Midl" /> 
    </contentFiles> 
   </metadata> 
   <files>
     <!--pp file-->
    <file src="ExamplePP.cpp.pp" target="contentFiles\cpp\any" />
    <file src="ExampleC++.cpp" target="contentFiles\cpp\any" />
    <file src="ExampleC.c" target="contentFiles\cpp\any" />
    <file src="ExampleASM.asm" target="contentFiles\cpp\any" /> 
    <file src="ExampleRC.rc" target="contentFiles\cpp\any" />
    <file src="ExampleMIDL.idl" target="contentFiles\cpp\any" />
  </files> 
</package>

[FogDai]

ncie

[mingkuang-Chuyu]

RID is ok. This is the test configuration.

wimgapi ,10.0.18362.1.

win-x86,win-x64,win-arm,win-arm64

<package> 
  <metadata minClientVersion="3.3.0"> 
    <id>Wimgapi</id> 
    <version>10.0.18362.1</version> 
    <authors>nuget</authors> <!-- The NuGet team authored this package --> 
    <owners>nuget</owners> <!-- The NuGet team owns this package --> 
    <requireLicenseAcceptance>false</requireLicenseAcceptance> 
    <description>A content v2 example package.</description> 
    <tags>Wimgapi native</tags> 
    <contentFiles>
      <files include="cpp/**" buildAction="None" copyToOutput="true"/>
    </contentFiles>
   </metadata> 
  <files>
    <file src="SDKs\Wimgapi\**" target="build\native"/>
    
    <file src="amd64\DISM\wimgapi.dll" target="runtimes\win-x64\native\wimgapi.dll"/>
    <file src="amd64\DISM\wimmount.sys" target="runtimes\win-x64\native\wimmount.sys"/>
    <file src="amd64\DISM\wimmountadksetupamd64.exe" target="runtimes\win-x64\native\wimmountadksetupamd64.exe"/>
    <file src="amd64\DISM\wofadk.sys" target="runtimes\win-x64\native\wofadk.sys"/>

    <file src="arm\DISM\wimgapi.dll" target="runtimes\win-arm\native\wimgapi.dll"/>
    <file src="arm\DISM\wimmount.sys" target="runtimes\win-arm\native\wimmount.sys"/>
    <file src="arm\DISM\wimmountadksetuparm.exe" target="runtimes\win-arm\native\wimmountadksetuparm.exe"/>
    <file src="arm\DISM\wofadk.sys" target="runtimes\win-arm\native\wofadk.sys"/>

    <file src="arm64\DISM\wimgapi.dll" target="runtimes\win-arm64\native\wimgapi.dll"/>
    <file src="arm64\DISM\wimmount.sys" target="runtimes\win-arm64\native\wimmount.sys"/>
    <file src="arm64\DISM\wimmountadksetuparm64.exe" target="runtimes\win-arm64\native\wimmountadksetuparm64.exe"/>
    <file src="arm64\DISM\wofadk.sys" target="runtimes\win-arm64\native\wofadk.sys"/>
    
    <file src="x86\DISM\wimgapi.dll" target="runtimes\win-x86\native\wimgapi.dll"/>
    <file src="x86\DISM\wimmount.sys" target="runtimes\win-x86\native\wimmount.sys"/>
    <file src="x86\DISM\wimmountadksetupx86.exe" target="runtimes\win-x86\native\wimmountadksetupx86.exe"/>
    <file src="x86\DISM\wofadk.sys" target="runtimes\win-x86\native\wofadk.sys"/>


    <file src="amd64\DISM\de-de\wimgapi.dll.mui" target="contentFiles\cpp\native\de-de\wimgapi.dll.mui"/>
    <file src="amd64\DISM\en-us\wimgapi.dll.mui" target="contentFiles\cpp\native\en-us\wimgapi.dll.mui"/>
    <file src="amd64\DISM\es-es\wimgapi.dll.mui" target="contentFiles\cpp\native\es-es\wimgapi.dll.mui"/>
    <file src="amd64\DISM\fr-fr\wimgapi.dll.mui" target="contentFiles\cpp\native\fr-fr\wimgapi.dll.mui"/>
    <file src="amd64\DISM\it-it\wimgapi.dll.mui" target="contentFiles\cpp\native\it-it\wimgapi.dll.mui"/>
    <file src="amd64\DISM\ja-jp\wimgapi.dll.mui" target="contentFiles\cpp\native\ja-jp\wimgapi.dll.mui"/>
    <file src="amd64\DISM\ko-kr\wimgapi.dll.mui" target="contentFiles\cpp\native\ko-kr\wimgapi.dll.mui"/>
    <file src="amd64\DISM\pt-br\wimgapi.dll.mui" target="contentFiles\cpp\native\pt-br\wimgapi.dll.mui"/>
    <file src="amd64\DISM\ru-ru\wimgapi.dll.mui" target="contentFiles\cpp\native\ru-ru\wimgapi.dll.mui"/>
    <file src="amd64\DISM\zh-cn\wimgapi.dll.mui" target="contentFiles\cpp\native\zh-cn\wimgapi.dll.mui"/>
    <file src="amd64\DISM\zh-tw\wimgapi.dll.mui" target="contentFiles\cpp\native\zh-tw\wimgapi.dll.mui"/>

    <file src=".\Wimgapi.targets" target="build\native\Wimgapi.targets" />
  </files>
</package>

[mingkuang-Chuyu]

Yesterday I added vcxproj's targetFramework support, with the help of targetFramework. We can configure different static or dynamic libraries for different toolsets.

targetFramework matching order

  (vs2019)
native v14.20.27508, 14.21.27702 ...
      |
  (vs2017)
native v14.10.25017, 14.11.25503 ...
      |
  (vs2015)
native v14.0
      | 
native v0.0 

For example, if the following 2 paths exist in a package:

• build\native14.0\Test.Static.targets
• build\native14.20\Test.Static.targets

If we use the v142 toolset (VCToolsVersion = 14.23.28105 vs2019), then build\native14.20\Test.Static.targets will insert * .nuget.g.targets.

If we use the v141 toolset (14.12.25827 vs2017), then build\native14.0\Test.Static.targets will insert * .nuget.g.targets.

in nuspec

in nuspec, using targetFramework, we can configure different static libraries for different toolsets of C++ projects.

    <dependencies>
      <group targetFramework="native14.0">
        <dependency id="Detours.Static.v140" version="4.0.1" />
      </group>
      <group targetFramework="native14.10">
        <dependency id="Detours.Static.v14.10" version="4.0.1" />
      </group>
      <group targetFramework="native14.20">
        <<dependency id="Detours.Static.v14.20" version="4.0.1" />
      </group>
      <group targetFramework="native14.21.27702">
        <<dependency id="Detours.Static.v14.21.27702" version="4.0.1" />
      </group>
    </dependencies>

in vcxproj

we can also do like this:

  <ItemGroup>
    <PackageReference Include="Detours.Static.v14.13.26128" Condition=" '$(TargetFramework)' == 'native14.13.26128' ">
      <Version>4.0.1</Version>
    </PackageReference>
  </ItemGroup>

[mingkuang-Chuyu]

I'm added VCToolsVersion support, Because VS allows you to choose different versions of the toolset, there are some differences between different versions. I supported this feature through targetFramework

native v14.20.27508, 14.21.27702 ...
                |
       native v14.20
                |
       native v14.0

[mingkuang-Chuyu]

I modified the contentFiles support. In CreateItem, I added AdditionalMetadata = "PrecompiledHeader = NotUsing".

dotnet/NuGet.BuildTasks@812a5d9#diff-5b809cf81a6b552123b02a4b102867c9

The PrecompiledHeader is unique to C++ projects, and this option reduces the generality of the asset. Because we don't know if the project has PrecompiledHeader enabled. And we don't know the name of the precompiled header file.

For other projects adding PrecompiledHeader, nothing will happen.

[davidhunter22]

Hi, I'm not an expert just a user but am wondering if this will allow a vcxproj style project to PackageReference a C# package that is build AnyCPU?

[mingkuang-Chuyu]

Hi, I'm not an expert just a user but am wondering if this will allow a vcxproj style project to PackageReference a C# package that is build AnyCPU?

VC++ does not support AnyCPU.
: )

[davidhunter22]

Sorry I should have been cleared a C++/CLI project that is built with /clr and therefore produces IL. At present projects like that can consume "native" nuget packages but I have never found a way to make them consume .NET packages.

[mingkuang-Chuyu]

Sorry I should have been cleared a C++/CLI project that is built with /clr and therefore produces IL. At present projects like that can consume "native" nuget packages but I have never found a way to make them consume .NET packages.

I'll try to support this feature (that C++/CLI project use .NET nuget package ).

[mingkuang-Chuyu]

Sorry I should have been cleared a C++/CLI project that is built with /clr and therefore produces IL. At present projects like that can consume "native" nuget packages but I have never found a way to make them consume .NET packages.

Sorry NuGet does not support TargetFramework Fallblack, so I cannot support them at the same time. Even if I provide PR, but there are many changes, I am afraid that Microsoft's urine is difficult to accept.

[rrelyea]

@AugP from Microsoft's C++ team commented on a related PR (NuGet/NuGet.Client#3145 (comment)):

Hello and thank you for the contribution! I work on the C++ team. First I want to apologize for the delay in responding - we have been actively monitoring PackageReference feedback as well as this PR, and we should have been more communicative.

From our side, here is the situation:

We are working on plans to improve the C++ package management story. Specifically, we want to extend the features of our existing C++ package manager, vcpkg: https://github.com/microsoft/vcpkg, and improve build system integration, but across multiple build systems, not just MSBuild. We also want vcpkg to be able to support existing NuGet-packaged libraries in some way.
Our concern with going all in on PackageReference is that it an MSBuild-specific concept, which would not be extensible to CMake projects or other external build systems. Within the next year or less, we want to provide the means to attach any C++ library to any of the project systems we support. We are also considering ways to provide MSVC toolsets and other build-time tools as NuGet packages, and are determining that we would want something more powerful than PackageReference to accomplish this (and again we would want to be build system agnostic).
If we accept this PR and do any additional work required on the IDE side to unblock PackageReference usage for C++, the work would be rendered obsolete in the future when we move to the build system agnostic model. Our concern there is that if developers do the work to migrate to C++ PackageReference, they would later realize that we don't intend to provide ongoing support and want them to move to something else. I expect that would be a frustrating experience.
So our team is not decided on whether or not to add broad PackageReference support right now. If we do, we would be resolving the issue more quickly for the many people who are looking for this feature. But we would see the solution more like a band-aid solution than a complete one, and the NuGet C++ experience would still suffer in a few ways, as it does today for packages.config:

Automatic restore on project load would not work
Our project caching system would not work at the time a NuGet restore is done (impacts solution load times)
Both of those issues could be fixed in theory, but we would likely shift our resources instead to the bigger solution we have been planning as it would benefit more developers.

For now, we are definitely committing to PackageReference support for C++/CLI projects targeting .NET Core, as there are cases there where developers are required to pull in NuGet packages in a workflow where packages.config no longer works at all.

I am continuing conversations with my team to determine if broader PackageReference support makes sense as a path and will keep you posted.

Given this, we're closing that PR, and keeping this issue open while the C++ teams conversations continue.'

Note...the C++ is most engaged on this developer community item about this same topic. See https://developercommunity.visualstudio.com/idea/351636/use-packagereference-in-vcxproj.html