Ignore shared Collective mounts when scanning files

[arnowelzel]

• Resolves: [Bug]: Shared Collectives cause "Path not found" when scannig files with "occ files:scan --all" #42496

Summary

When scanning files using occ files:scan, shared Collective mounts trigger "Path not found" error messages. This change will make sure that that Collective mounts will be ignored.

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[arnowelzel]

/backport to stable28

[emoral435]

Nice change, looks good to me :)

[emoral435]

Fast review ~!

[artonge]

Looks good, but let's wait for @max-nextcloud or @mejo- input :)

[max-nextcloud]

@mejo- is the better person when it comes to backend changes.

I have some uninformed questions though...

• What will happen if collectives is not installed and the used class is not available?
  Will php be happy with that?
• You are talking about a shared collective - are you referring to a collective that has been explicitly shared by link or a normal collective with multiple members?

In general it seems not ideal to me to have collectives specific code in server. I wonder if we could instead make CollectiveStorage a SharedStorage as that could happen on the collectives side of things.

[max-nextcloud]

I remember that collectives storage was designed closely to what groupfolders does. Does this issue not occur with groupfolders ? How is the groupfolders implementation different from collectives?

[arnowelzel]

@mejo- is the better person when it comes to backend changes.

I have some uninformed questions though...

• What will happen if collectives is not installed and the used class is not available?
  Will php be happy with that?

I tested this case and had no problem without Collectives installed, since instanceOfStorage() will only check the class name and it should not matter, if the class actually exists.

But I agree it would be better not to rely on multiple classes here but instead have a common method which can be used. Maybe adding a method isShared() to the Storage interface may help. This sould return true by SharedStorage or CollectiveStorage and otherwise false so the scanner knows when to ignore the storage:

			// don't scan shared storages, these can be scanned when scanning the owner's storage
			if ($storage->isShared()) {
				continue;
			}

However this would involve substantial refactoring of the backend code.

• You are talking about a shared collective - are you referring to a collective that has been explicitly shared by link or a normal collective with multiple members?

I refer to a normal collective which is shared with multiple members of a circle.

[skjnldsv]

@max-nextcloud @mejo- any update?

[skjnldsv]

@arnowelzel psalm said no

[arnowelzel]

Since using the class name of the Collective storage is not possible in any way without triggering a psalm error (even when the class exists it is not allowed as parameter for \OC\Files\Storage\Wrapper\Wrapper::instanceOfStorage()) I changed the test as following:

			// don't scan received local shares or collectives, these can be scanned when scanning the owner's storage
			if (substr($storage->getId(), 0, 6) !== 'home::') {
				continue;
			}

The idea here is, that only home storages should get scanned. Recieved shared storage start with shared:: and Collectives, which trigger the error, report IDs as local:: followed by the physical path. However I don't know if any other storages which should indeed be scanned, also report an ID starting with something different from home::.

I also noticed the following code in \OCA\Collectives\Mount\CollectiveStorage - maybe in the past this problem was mitigated by using NoopScanner which does not exist any longer?

	public function getScanner($path = '', $storage = null): Scanner {
		if (!$storage) {
			$storage = $this;
		}
		if ($storage->instanceOfStorage(ObjectStoreStorage::class)) {
			// NoopScanner is private API and kept here for compatibility with older releases
			$storage->scanner = class_exists(NoopScanner::class) ? new NoopScanner($storage) : new ObjectStoreScanner($storage);
		} elseif (!isset($storage->scanner)) {
			$storage->scanner = new Scanner($storage);
		}
		return $storage->scanner;
	}

[mejo-]

Probably @icewind1991 is the best to decide whether it's a good idea to limit scanning to home storages.

[icewind1991]

This will break scanning of external storages.

[arnowelzel]

What ID do external storages have?

[icewind1991]

they all have different prefixes, in general attempts to filter by storage id is imprecise at best.

[icewind1991]

What is stopping the collectives from being scanned correctly in the first place?

[arnowelzel]

What is stopping the collectives from being scanned correctly in the first place?

They are shared storages like SharedStorage and thus don't provide a physical path which can be scanned.

[icewind1991]

If they can't be scanned at all then having them use NoopScanner is probably the best (shared storage should also move to that really)

[arnowelzel]

If they can't be scanned at all then having them use NoopScanner is probably the best (shared storage should also move to that really)

Well - NoopScanner does not exist any longer.

[icewind1991]

Right, it got moved to ObjectStoreScanner when it got slightly less noop, should be good to still use that one or just create your own

[arnowelzel]

Right, it got moved to ObjectStoreScanner when it got slightly less noop, should be good to still use that one or just create your own

It seems, this is what Collectives is using right now - but it does not work. See https://github.com/nextcloud/collectives/blob/main/lib/Mount/CollectiveStorage.php#L62-L74:

public function getScanner($path = '', $storage = null): Scanner {
		if (!$storage) {
			$storage = $this;
		}
		if ($storage->instanceOfStorage(ObjectStoreStorage::class)) {
			// NoopScanner is private API and kept here for compatibility with older releases
			$storage->scanner = class_exists(NoopScanner::class) ? new NoopScanner($storage) : new ObjectStoreScanner($storage);
		} elseif (!isset($storage->scanner)) {
			$storage->scanner = new Scanner($storage);
		}
		return $storage->scanner;
	}