Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving Filestructure of preview folder #14169

Closed
PichlAlex opened this issue Feb 12, 2019 · 1 comment
Closed

Improving Filestructure of preview folder #14169

PichlAlex opened this issue Feb 12, 2019 · 1 comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap enhancement

Comments

@PichlAlex
Copy link

the Issue that i adress to
I am running a huge Nextcloud installation (not in filesize, but in filecount) in private use. Currently i have around 1.000.000 Files hosted on the nextcloud instance, across 4 Users. There are Issues accessing the previews (lots of disk IO when loading a single preview and viewing a listing page gets really slow) because of the foldercount in nextcloud/data/appdata_xxxx/preview/
Inside this folder i currently have ~1,3 Million folders with multiple previews in each -> 300 GB

I am using Nextcloud on ubuntu Server 16.04 on BTRFS and the Previews are stored on Raid10 7200rpm disks

Describe the solution you'd like
i would recommend to store all the folders through subfolders that in each single folder the number of Folders is limited to eg 1000 folders that still can be quickly accessed and does not cause a lot of issues when accessing.
eg - Variant 1
preview/0/0/0/0 stores images 1 to 999
preview/0/0/0/1 stores images 1000 to 1999
preview/0/0/1/0/ stores images 10000-10999
preview/0/0/5/6 stores images 56000 - 569999

eg Variant 2:
preview/1/ stores images 1 to 999
preview/1000 stores images 1000 to 1999
preview/56000 stores images 56000 to 56999
...

Describe alternatives you've considered
maybe another storing method should be applied (by hash of filelocking? (and use a kind of Variant 2)
https://help.nextcloud.com/t/info-how-the-oc-file-locks-table-actually-works/28727 )

@MorrisJobke
Copy link
Member

Was fixed in 19 with #19214

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap enhancement
Projects
None yet
Development

No branches or pull requests

2 participants