Updating repositories fails on Linux if /tmp/CdFileMgr is non-writable

[keyspace]

CKAN Version
1.16.1-0-g2e91715 (current master) or 1.16.1 (tried both) - building and running with mono.

Operating System
Arch Linux 64-bit (although expect to happen on all GNU/Linux variants).

The issue
A previous run of CKAN by a different user left the temporary directory, and the permissions on it disallow another user from writing master.tar.gz.

How to recreate this issue

• Have a multi-user system setup, such as data segregation for development purposes or separate users for family members.
• Run by user1 and exit.
• Run by user2.

Workarounds

Either of:

• Delete old dir.
• Change permissions on dir to be writable by user2.

Proposed fix

• Use a unique identifier based on UID, PID, or just a plain pseudo-random number.

[politas]

Seems like a simple solution. I can't think of a reason it would fail. Only issue is that I can't find a string "CdFileMgr" anywhere in the CKAN code, so maybe that's in Mono system code.

[ayan4m1]

The temp file name appears to come from the TxFileManager class.

Source of the file is here

[keyspace]

Probably should re-open, no indication it's been fixed?

[ayan4m1]

This is likely an intended feature of the file transaction library. There are many forks of it, but the second link here is the package we are referencing when building CKAN:

https://github.com/rsevil/Transactions/blob/master/src/ChinhDo.Transactions.FileManager/Utils/FileUtils.cs#L8

http://transactionalfilemgr.codeplex.com/SourceControl/latest#ChinhDo.Transactions.FileManager/FileUtils.cs

I think this is an upstream issue, and this library isn't being actively maintained that I can tell.

[politas]

Is there something we could do to release the temp working directory so it gets deleted?

[ayan4m1]

Sure, we could do something like that, but devil's advocate - we are now writing code into CKAN that makes "ChinDo.Transactions" a hardcoded dependency because the "CdFileMgr" name comes straight from their source and is a unique issue when combining an old lib with an edge case in Mono. As of right now I don't like this lib just because it doesn't seem designed with Linux/Mono in mind despite it seeming like the primary platform for this app. Since it hasn't been updated since 2013 why would it have though... so my opinion is moving to a newer version of this library like this one would resolve the issue without any other changes required. I can get into this further in a few days based on team thoughts.

[politas]

That rsevil fork looks like it was all set up for replacing the copy on Nuget, but has clearly never been accepted by the upstream project, and hasn't had any updates since that short flurry of mostly refactoring and doco changes in Dec 2014. It does indeed appear to be an abandoned project. I suppose our options come down to

• using that random source (or some other fork) for a single update,
• taking our own fork of the code (which is presumably going to be rather complex to manage),
• finding a way to work around the issue using the main code, or
• finding a replacement that's a more active project.

I'd want a pretty serious code review of the rsevil or other fork if we take that route. Taking our own fork of the codebase would let us prevent future malicious changes, but mean we have to handle the complexities. My quick Google search finds absolutely no alternates for the latter option.

[HebaruSan]

The original project shows recent signs of life (8 commits in the past 2 weeks), and no longer seems to contain the string "CdFileMgr":

• https://github.com/chinhdo/txFileManager
• https://www.nuget.org/packages/TxFileManager/

Might be worth updating our nuget dependency from ChinhDo.Transactions.FileManager to TxFileManager.

[HebaruSan]

Hmm, dead end?

NU1202: Package TxFileManager 1.4.0 is not compatible with net45 (.NETFramework,Version=v4.5). Package TxFileManager 1.4.0 supports: netstandard2.0 (.NETStandard,Version=v2.0)

1.3.0 looks OK though... except it still creates /tmp/CdFileMgr :(

[HebaruSan]

As of #4102, temp files are no longer used for repo updates.