Add automatic cleanups to improve the inner devloop

[kzu]

When working with packages built locally, it's quite common to dogfood those packages from sample projects locally. This can use a fixed package version for the locally produced packages, or auto-incremented versions. Also, the referencing projects might choose to use wildcards when testing local packages too.

The caching mechanisms built into NuGet make this process a bit more cumbersome than necessary: if you build a fixed version package, you will never get a newly built version restored in a project elsewhere in the machine because NuGet will believe the one in the cache is already the latest. The HTTP-level cache implemented on top of the package cache also works against you in that case even if you clean that folder. And wildcards don't make things much better unless you clean those caches too.

In addition, if you increment package versions when building locally too, the package output path will continuously be filled up with older versions unnecessarily.

This commit adds support for automatically fixing all those issues while still causing minimal disruption or performance problems for other packages and projets in your machine, as follows:

• The entire cleanup only is in place for packable projects, and in local (non-CI) builds
• It can be turned off entirely by setting EnablePackCleanup=false.
• It cleans the specific package folder in the cache for the current PackageId: nuget creates a subfolder in the package cache dir for each package id, and places all versions inside. By removing just that folder, you effectively clean the cache for that package and no others.
• It cleans the HTTP cache too: this cannot be done selectively for a specific package id, and therefore can be turned off by setting CleanHttpNuGetCacheOnPack=false if it causes performance issues. In my experience, it doesn't since the HTTP cache is just an optimization for offline scenarios (I think?).

I have used this approach for years on multiple projects with multiple packaging approaches and at this point I think it deserves being built-in in nugetizer.

[Aleksanderis]

@kzu , sorry for necro-posting, but I have a direct question related to inner devloop.
I'm happy how it works during development, how it cleans caches, etc. But what is best way to revert back to "normal" behavior? So that would consume officially published packages again, not locally built ones?

Maybe I missed something and it's documented somewhere, or maybe it's just to obvious, therefore not documented? :)
The only idea I have, is to execute "pack" on the library, so that it would do its' magic to clean up caches once again, BUT then I clean up local output of it, BEFORE going to restore packages in consuming project. It kind of works, but sometimes it gets stuck, and it's cumbersome in general. ;)

Some real case scenario:

• I'm working myself on some NuGet library, build and debug it locally.
• I finish my job and publish everything to remote repository, and forget about the library.
• Later on someone else works on same NuGet library and produces new changes.
• I would like open the project where it's consumed, and just restore all NuGet packages, without going into "inner devloop" with the package.

[kzu]

Not sure I understand the scenario. Perhaps you can create a new issue and explain what works and doesn't in the current flow? (and what's your ideal behavior).

You can turn off all the cleanup by just setting EnablePackCleanup=false in your project, btw. Everything in NuGetizer can be turned off via simple properties :)

[Aleksanderis]

I think I managed to workaround it, but had to deviate a bit from what was suggested in documentation.
I can describe briefly how I solved it, maybe it can be useful for others.

1. First issue was related to versioning. Our publicly released NuGet packages are versioned autoincrementally on every build. It was causing conflicts when locally (in inner devloop) 1.x.x packages were produced. It kind of was trying to get the only existing packages, but during the builds it was throwing errors about version mismatch. I'm not sure - maybe it was a warning, but we just have "WarningAsErrors=true". To workaround it - I added <PackageVersion>x.x.x</PackageVersion> to my Directory.Builds.props in solution with the packages. So everytime I want to go into "inner devloop", I have just to update this property to match with latest published packages - it's a manual thing, but it's not needed to be done often. (During CI build package version is set via build parameters, therefore version specified in Directory.Builds.props is ignored.)

2. Another issue was with <RestoreSources></RestoreSources> mentioned in documentation (https://github.com/devlooped/nugetizer#inner-devloop).

Instead of this:

<RestoreSources>https://api.nuget.org/v3/index.json;$(RestoreSources)</RestoreSources>
<RestoreSources Condition="Exists('$(MSBuildThisFileDirectory)..\..\bin\')">
  $([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)..\..\bin'));$(RestoreSources)
</RestoreSources>

I ended with this (removed $(RestoreSources) from the second property):

<RestoreSources>https://api.nuget.org/v3/index.json;$(RestoreSources)</RestoreSources>
<RestoreSources Condition="Exists('$(MSBuildThisFileDirectory)..\..\bin\')">
  $([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)..\..\bin'));
</RestoreSources>

Not sure why it works like that, but if I leave it like in documentation - from time to time it was restoring packages from the remote repository anyway.

3. Third thing to do - when I'm done with "inner devloop" and want to get back consuming packages from remote repositories - I just go and delete '$(MSBuildThisFileDirectory)..\..\bin\' the folder. When it's empty - packages are restored from the remote repository.

So, overall it still not a flawless development experience - some manual steps are involved, also messing up with NuGet caches often makes Rider/VisualStudio to act up and scream about broken references temporarily (some additional pressure on CPU/Memory, etc). However it's still way better than everytime publishing packages to remote repository when you want just to test something. I can live with it. :)

[kzu]

1. I always use 42.42.42 as the local version. It's pointless to have any meaningful version number when developing locally. By making it a large number, it will always be greater than any published version in nuget.org. I also update references via targets so they point to that version if found locally. Here's a targets I frequently set up (i.e. as Directory.Build.targets):

<Project>
	<PropertyGroup>
		<ThisAssemblyDir>C:\Code\ThisAssembly\bin</ThisAssemblyDir>
		<RestoreSources Condition="Exists('$(ThisAssemblyDir)')">$(ThisAssemblyDir);$(RestoreSources)</RestoreSources>

		<NuGetizerDir>C:\Code\nugetizer\bin</NuGetizerDir>
		<NuGetizerDev Condition="Exists('$(NuGetizerDir)\NuGetizer.42.42.42.nupkg')">true</NuGetizerDev>
		<RestoreSources Condition="'$(NuGetizerDev)' == 'true'">$(NuGetizerDir);$(RestoreSources)</RestoreSources>

		<GitInfoDir>C:\Code\GitInfo\bin</GitInfoDir>
		<GitInfoDev Condition="Exists('$(GitInfoDir)\GitInfo.42.42.42.nupkg')">true</GitInfoDev>
		<RestoreSources Condition="'$(GitInfoDev)' == 'true'">$(GitInfoDir);$(RestoreSources)</RestoreSources>
	</PropertyGroup>

	<ItemGroup Condition="Exists('$(ThisAssemblyDir)')">
		<PackageReference Update="ThisAssembly.AssemblyInfo" Version="42.42.42" />
		<PackageReference Update="ThisAssembly.Constants" Version="42.42.42" />
		<PackageReference Update="ThisAssembly.Metadata" Version="42.42.42" />
	</ItemGroup>

	<ItemGroup>
		<PackageReference Condition="'$(NuGetizerDev)' == 'true'" Update="NuGetizer" Version="42.42.42" />
		<PackageReference Condition="'$(GitInfoDev)' == 'true'" Update="GitInfo" Version="42.42.42" />
	</ItemGroup>
</Project>

2. That's a side-effect of you trying to use version numbers that are in the public feed. 1) fixes that. If you don' have the upstream feed, things will break if you add a package that's not already in the cache.
3. Same thing I do, yep.

[Aleksanderis]

Thanks for more input, I will try with updating references via targets.
Yes, I tried using "random" local version, which was always higher than published one. It wasn't enough - as I mentioned, code was resolving all the references, etc, but it was failing during build (which might be was just a warning), so updated references probably was a missing point. Maybe it's worth to mention it in the documentation? :)