One Cache to Rule Them All

[HebaruSan]

Background

CKAN stores downloads in a cache, so if you install a module twice, it won't be downloaded twice. Currently this cache lives at the game-instance level. Let's say for the sake of argument that a user had several game instances:

Then each of these would have its own independent download cache, at <GameRoot>/CKAN/downloads.

Problems

• If you install a module in your 1.2.2 instance, and then install the same module in your 1.2.2_Dakar instance, it would be downloaded twice and stored on disk twice.
• The cache can grow quite big and takes up space on the same disk/partition where KSP is installed with no option for configuration (other than manually arranged symlinks on Linux)

Three Caches for the Elven-kings under the sky,
Seven for the Dwarf-lords in halls of stone,
Nine for Mortal Men, doomed to die,

... is a bit much for CKAN.

Previous approaches

#2185 started addressing this, but it had some problems and @Olympic1 has been too busy to develop it further. This pull request tries to learn from and avoid the issues that came up in that effort.

• ckan help didn't show the cache command
• All instances' registry.json files were loaded at startup, causing a big performance hit
• Some of the tests didn't work because 0000400 isn't an octal literal in C#
• The cache folder configuration was stored in several places (Windows registry, CKAN instance registry, GUIConfig.xml file), which is kind of awkward to coordinate and opens the door to things getting out of sync

Changes

Core

The cache object is removed from the game-instance level altogether; the KSP class no longer has a Cache property (though it retains its cache folder property). Instead the cache now lives in KSPManager, the object that sits above all the instances. This makes it a truly global cache.

Various places have been updated to pass a cache or manager object where an instance would have worked previously.

Cache size measurement and clearing are migrated into the cache object itself.

The cache writes new downloads into one configurable location.The default cache location is:

Path.Combine(
    Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
    "CKAN",
    "downloads"
);

• On Linux this is ~/.local/share/CKAN/downloads
• On Windows it is C:\Users\<user>\AppData\Local\CKAN\downloads

This can be overridden by using the various UIs to set a Windows/Mono registry key:

KEY_CURRENT_USER\Software\CKAN\DownloadCacheDir

This allows the user to select a location on their system with the appropriate combination of capacity and responsiveness, according to their preferences. When you change the setting, files are migrated from the old folder to the new.

If the user has old cache folders in his existing game instances, these will be checked after the main cache folder but not written to. This ensures that previously downloaded files will not be re-downloaded regardless of the instances in which they were originally downloaded or newly needed. The user is free to migrate or purge these folders or leave them alone as he pleases. By not auto-migrating the contents of such folders, we avoid the risk of filling a disk when moving lots of big files from one folder to another.

Newly configured game instances will not have a CKAN/downloads folder created.

Netkan

Netkan is updated to use the configurable global cache. The --cachedir parameter still overrides the default path, so the bot should not be affected.

CmdLine

CmdLine now has several new subcommands (which show up in ckan help):

• ckan cache list - show the current cache location and size
• ckan cache set <path> - change the cache location
• ckan cache clear - delete all the files in the cache
• ckan cache reset - set the cache location back to the default

GUI

The GUI settings window's cache section is re-done:

• The cache path field is editable (this is necessary because the folder select window hides dot-folders, see Show dot-folders in folder browser mono/mono#10320)
• The Change... button opens a folder select window to choose a new path folder
• The Reset button sets the path back to the default, in case the user messes up the field and wants a do-over
• The Open button opens the cache in the operating system's file manager

If you type an invalid folder, the text description between the field and the buttons turns red and explains the problem (and the most recent valid selection will be used if the window is closed):

There is no "nag" feature on startup. Rather, the new cache structure will "just work" without the user having to worry about the details, but if they want to change it, it's there in the settings.

Fixes #584.
Fixes #960.
Closes #2185.

[Gryffen1971]

Now that is a very good idea. Very well thought out. Nice Job.

[Olympic1]

Thanks for continuing this

[politas]

This wasn't supposed to delete every file from my extensive cache, was it?

[HebaruSan]

(this is necessary because the folder select window hides dot-folders, see mono/mono#10320)

The Mono team have decided to let the perfect be the enemy of the good, sigh.