(#3231) Don't refresh local package info during upgrade no-ops

[TheCakeIsNaOH]

Description Of Changes

This optimizes the flow of the upgrade command. First, it uses the
list of locally installed package from the if all is specified check,
instead of throwing that away. Then, it will re-use the list for each
package upgrade attempt, only refreshing the list when a package
upgrade/install is attempted. This speeds up upgrade all runs,
especially when a large (200+) number of packages are installed.

Motivation and Context

Upgrades on my machine are really slow as compared with their speed on v1.4.0

Testing

1. Clean the debug directory, then build debug choco from develop
2. Create this packages.config file:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="7zip" />
  <package id="7zip.install" />
  <package id="adobeair" />
  <package id="adobereader" />
  <package id="adobereader-update" />
  <package id="adobeshockwaveplayer" />
  <package id="advanced-installer" />
  <package id="advanced-ip-scanner" />
  <package id="amd-ryzen-chipset" />
  <package id="atom" />
  <package id="atom.install" />
  <package id="audacity" />
  <package id="autohotkey.install" />
  <package id="autohotkey.portable" />
  <package id="autoruns" />
  <package id="awscli" />
  <package id="azure-cli" />
  <package id="azure-pipelines-agent" />
  <package id="balcon" />
  <package id="Boxstarter" />
  <package id="Boxstarter.Bootstrapper" />
  <package id="Boxstarter.Chocolatey" />
  <package id="Boxstarter.Common" />
  <package id="Boxstarter.HyperV" />
  <package id="Boxstarter.WinConfig" />
  <package id="brave" />
  <package id="calibre" />
  <package id="ccleaner" />
  <package id="chef-client" />
  <package id="chocolatey-compatibility.extension" />
  <package id="chocolatey-core.extension" />
  <package id="chocolatey-dotnetfx.extension" />
  <package id="chocolatey-misc-helpers.extension" />
  <package id="chocolatey-os-dependency.extension" />
  <package id="chocolatey-uninstall.extension" />
  <package id="chocolatey-visualstudio.extension" />
  <package id="chocolatey-windowsupdate.extension" />
  <package id="citrix-receiver" />
  <package id="citrix-workspace" />
  <package id="crystaldiskinfo" />
  <package id="crystaldiskinfo.portable" />
  <package id="CrystalReports2008Runtime" />
  <package id="CrystalReports2010Runtime" />
  <package id="curl" />
  <package id="cutepdf" />
  <package id="Cygwin" />
  <package id="dellcommandupdate" />
  <package id="dell-update" />
  <package id="docker-desktop" />
  <package id="dotnet" />
  <package id="dotnet1.1" />
  <package id="dotnet3.5" />
  <package id="DotNet4.0" />
  <package id="dotnet4.5" />
  <package id="DotNet4.5.1" />
  <package id="dotnet4.5.2" />
  <package id="DotNet4.6" />
  <package id="dotnet4.6.1" />
  <package id="dotnet4.6.2" />
  <package id="DotNet-4.6.2" />
  <package id="dotnet4.7" />
  <package id="dotnet4.7.1" />
  <package id="dotnet4.7.2" />
  <package id="dotnet-5.0-desktopruntime" />
  <package id="dotnet-5.0-runtime" />
  <package id="dotnet5-desktop-runtime" />
  <package id="dotnet-6.0-desktopruntime" />
  <package id="dotnet-6.0-runtime" />
  <package id="dotnet-7.0-desktopruntime" />
  <package id="dotnet-7.0-runtime" />
  <package id="dotnet-all" />
  <package id="dotnet-aspnetcoremodule-v2" />
  <package id="dotnetcore" />
  <package id="dotnetcore-2.0-runtime" />
  <package id="dotnetcore-2.1-runtime" />
  <package id="dotnetcore-2.2-runtime" />
  <package id="dotnetcore-3.0-desktopruntime" />
  <package id="dotnetcore-3.0-runtime" />
  <package id="dotnetcore-3.1-aspnetruntime" />
  <package id="dotnetcore-3.1-desktopruntime" />
  <package id="dotnetcore-3.1-runtime" />
  <package id="dotnetcore-3.1-sdk" />
  <package id="dotnetcore-3.1-sdk-4xx" />
  <package id="dotnetcore-3.1-windowshosting" />
  <package id="dotnetcore3-desktop-runtime" />
  <package id="dotnetcore-desktopruntime" />
  <package id="dotnetcore-runtime" />
  <package id="dotnetcore-runtime.install" />
  <package id="dotnetcore-sdk" />
  <package id="dotnetcore-windowshosting" />
  <package id="dotnet-desktopruntime" />
  <package id="dotnetfx" />
  <package id="dotnet-runtime" />
  <package id="dropbox" />
  <package id="Everything" />
  <package id="ffmpeg" />
  <package id="filebeat" />
  <package id="filezilla" />
  <package id="Firefox" />
  <package id="FirefoxESR" />
  <package id="flashplayeractivex" />
  <package id="flashplayerplugin" />
  <package id="flashplayerppapi" />
  <package id="foxitreader" />
  <package id="Ghostscript.app" />
  <package id="gimp" />
  <package id="git" />
  <package id="git.install" />
  <package id="golang" />
  <package id="GoogleChrome" />
  <package id="googledrive" />
  <package id="google-drive-file-stream" />
  <package id="gotomeeting" />
  <package id="graphviz" />
  <package id="gsuite-sync-outlook" />
  <package id="icinga2" />
  <package id="imagemagick.app" />
  <package id="InkScape" />
  <package id="intel-dsa" />
  <package id="irfanview" />
  <package id="iTunes" />
  <package id="javaruntime" />
  <package id="jdk8" />
  <package id="jq" />
  <package id="jre8" />
  <package id="KB2533623" />
  <package id="kb2919355" />
  <package id="kb2919442" />
  <package id="kb2999226" />
  <package id="kb3033929" />
  <package id="kb3035131" />
  <package id="KB3063858" />
  <package id="KB3118401" />
  <package id="keepass" />
  <package id="keepass.install" />
  <package id="k-litecodecpackfull" />
  <package id="kubernetes-cli" />
  <package id="libreoffice-fresh" />
  <package id="llvm" />
  <package id="logrotate" />
  <package id="make" />
  <package id="malwarebytes" />
  <package id="maven" />
  <package id="metricbeat" />
  <package id="microsoftazurestorageexplorer" />
  <package id="microsoft-build-tools" />
  <package id="microsoft-edge" />
  <package id="microsoft-office-deployment" />
  <package id="microsoft-teams" />
  <package id="microsoft-teams.install" />
  <package id="microsoft-windows-terminal" />
  <package id="netfx-4.6.2" />
  <package id="netfx-4.7.1" />
  <package id="netfx-4.7.1-devpack" />
  <package id="netfx-4.7.2" />
  <package id="netfx-4.8" />
  <package id="nextcloud-client" />
  <package id="ninja" />
  <package id="nodejs" />
  <package id="nodejs.install" />
  <package id="nodejs-lts" />
  <package id="notepadplusplus" />
  <package id="notepadplusplus.install" />
  <package id="nssm" />
  <package id="nuget.commandline" />
  <package id="nvidia-display-driver" />
  <package id="nxlog" />
  <package id="office365business" />
  <package id="office365proplus" />
  <package id="onedrive" />
  <package id="opencl-intel-cpu-runtime" />
  <package id="open-shell" />
  <package id="openssh" />
  <package id="openssl" />
  <package id="Opera" />
  <package id="osquery" />
  <package id="paint.net" />
  <package id="pdf24" />
  <package id="PDFCreator" />
  <package id="playnite" />
  <package id="postman" />
  <package id="powershell" />
  <package id="powershell-core" />
  <package id="powertoys" />
  <package id="procexp" />
  <package id="pswindowsupdate" />
  <package id="puppet-agent" />
  <package id="putty" />
  <package id="putty.install" />
  <package id="putty.portable" />
  <package id="python" />
  <package id="python2" />
  <package id="python3" />
  <package id="python311" />
  <package id="qpdf" />
  <package id="Quicktime" />
  <package id="ruby" />
  <package id="ruby.install" />
  <package id="Silverlight" />
  <package id="skype" />
  <package id="slack" />
  <package id="spotify" />
  <package id="sql2016-clrtypes" />
  <package id="sql2016-smo" />
  <package id="sql-server-express" />
  <package id="sql-server-management-studio" />
  <package id="sqlserver-odbcdriver" />
  <package id="strawberryperl" />
  <package id="sumatrapdf.install" />
  <package id="sysinternals" />
  <package id="teamviewer" />
  <package id="teamviewer.host" />
  <package id="telegraf" />
  <package id="terraform" />
  <package id="thunderbird" />
  <package id="treesizefree" />
  <package id="UrlRewrite" />
  <package id="vcredist140" />
  <package id="vcredist2008" />
  <package id="vcredist2010" />
  <package id="vcredist2012" />
  <package id="vcredist2013" />
  <package id="vcredist2015" />
  <package id="vcredist2017" />
  <package id="vim" />
  <package id="virtualbox" />
  <package id="visualstudio2017buildtools" />
  <package id="visualstudio2017-workload-vctools" />
  <package id="visualstudio2019buildtools" />
  <package id="visualstudio2019-workload-vctools" />
  <package id="visualstudio-installer" />
  <package id="vlc" />
  <package id="vlc.install" />
  <package id="vmware-tools" />
  <package id="vscode" />
  <package id="vscode.install" />
  <package id="webex-meetings" />
  <package id="Wget" />
  <package id="winlogbeat" />
  <package id="winrar" />
  <package id="winscp" />
  <package id="winscp.install" />
  <package id="winspy" />
  <package id="wireshark" />
  <package id="wsl2" />
  <package id="yarn" />
  <package id="youtube-dl" />
  <package id="yt-dlp" />
  <package id="zoom" />
  <package id="Zoom-Outlook" />
</packages>

3. Run .\choco.exe install .\packages.config --skip-powershell (Note, only an internal source will work for this, otherwise it will quickly hit the rate limits on CCR)
4. Run Measure-Command { .\choco.exe upgrade all --ignore-http-cache --source=https://community.chocolatey.org/api/v2 --skip-powershell }
5. Switch to this PR, and build debug choco from this branch
6. Run Measure-Command { .\choco.exe upgrade all --ignore-http-cache --source=https://community.chocolatey.org/api/v2 --skip-powershell } again
7. Compare the two times. I get a time of 383 seconds for the develop branch build, and a time of 32 seconds for the PR build of choco, so the speedup is by more than an order of magnitude. The support/1.x branch was 36 seconds for me, so the same general speed as this PR.

Operating Systems Testing

• Windows 10 22H2

Change Types Made

• Bug fix (non-breaking change).
• Feature / Enhancement (non-breaking change).
• Breaking change (fix or feature that could cause existing functionality to change).
• Documentation changes.
• PowerShell code changes.

Change Checklist

• Requires a change to the documentation.
• Documentation has been updated.
• Tests to cover my changes, have been added.
• All new and existing tests passed?
• PowerShell code changes: PowerShell v2 compatibility checked?

Related Issue

Fixes #3231

[TheCakeIsNaOH]

I'm not sure if this is the best implementation of this general idea, I'd welcome a second look at this. If it does look ok, then I should be able to get a test set up for it.

[corbob]

So history id 8 was with Chocolatey 2.1.0, and id 16 was with the build for this PR...

With regards to adding tests: What were you thinking for tests for these? Other than the missing tests, everything here looks good to me.

[TheCakeIsNaOH]

What were you thinking for tests for these?

Probably one test to check that a dependency upgrade gets detected/refreshed, and one test to check that a dependency install (via an upgrade) gets detected/refreshed, for when the parent package gets upgraded second.

[corbob]

@TheCakeIsNaOH I've force pushed to your branch after updating to the head of the develop branch. I've also removed that extra empty line from your commit. If you have an opportunity to take a look at the tests I added, it'd be appreciated.

I have run the new integration tests with a revert commit removing the fix, and the added test for the count of the --- Start of List --- messages does fail there with 5 entries instead of the expected 3.

[TheCakeIsNaOH]

LGTM

[TheCakeIsNaOH]

Thanks for getting tests added @corbob, they look good.

[corbob]

Thanks for getting this fixed up @TheCakeIsNaOH 👍