Versioning creates zillions of empty blob files

[hvelarde]

When using plone.app.iterate, each version of a content object with persistent tiles with blobs, will create a new empty blob into file system for each found blob and I will also break the CMFEditions Portal ZVC based Histories Storage.

If you're using this feature in your site you have to take special attention to the number of free inodes in your file system, as you can run out of them; use the df -i command to check it:

$ df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
rootfs                    10485760 8929045 1556715   86% /
udev                       2057472     272 2057200    1% /dev
tmpfs                      2058962     204 2058758    1% /run
/dev/disk/by-label/DOROOT 10485760 8929045 1556715   86% /
tmpfs                      2058962       1 2058961    1% /run/lock
tmpfs                      2058962       5 2058957    1% /run/shm

[hvelarde]

This is related to plone/plone.app.mosaic#80

[hvelarde]

This is also related to plone/Products.CMFEditions#28

[runyaga]

@hvelarde does this commit fix the zillion of files? dont see any activty post 9/17 commit

[hvelarde]

@runyaga I don't know, but I don't want to fix/patch that here because then we will have to maintain it.

according to @datakurre the actual issue is different: deepcopying a blob results on a new blob with zero data and this is what CMFEditions does by default; check the issues referenced above.

please keep communication channels open before working on issues to avoid rework.

BTW, happy new year!

[datakurre]

In short: When attribute is not handled by a dedicated plugin in CMFEditions, it gets deep copied using pickle + unpickle. By Googling around I've understood that it's a "known" limitation of ZODB that cloning blobs that way results in empty blobs. Plone only has support for versioning blobs defined in Archetypes or Dexterity (or its behaviors') schemas.

Unfortunately tiles are designed in a way that it's not trivial to figure out tile's schema from its annotation object. To fix this, we need some more generic way version blobs in annotation (not cause conflict with behaviors using annotation storage while doing that)..

[fredvd]

@hvelarde I'm about to upgrade a Plone 4 site with an older collective.cover where this might be an issue, I have a blobstorage with 2/3 of the 15000 files being size zero.

The only weird thing is that plone.app.iterate was for some reason installed, but never available to end users (for some reason the actions were never available because the condition check fails).

For me and others who find this issue later, did you do anything to clean up the many empty blob files in your site after you upgraded cover with the plone.app.tiles 1.1.0 dependency that solves creating new blob files?

[hvelarde]

@fredvd you have to pack the database; BTW, for some reason I still have empty files on the sites that had this problem; is not the same high number, but this is still an issue.

I highly recommend to install collective.revisionmanager and run it from time to time to get rid of orphan versions.