Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] NetKAN leaving stale tmp files around #3168

Closed
techman83 opened this issue Oct 9, 2020 · 2 comments · Fixed by #3169
Closed

[Bug] NetKAN leaving stale tmp files around #3168

techman83 opened this issue Oct 9, 2020 · 2 comments · Fixed by #3169
Labels
Bug Something is not working as intended Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Netkan Issues affecting the netkan data

Comments

@techman83
Copy link
Member

Problem

NetKAN is leaving around stale files in CdFileMgr, leading to disk usage issues.

netkan@945a9d40d33e:/tmp/CdFileMgr$ du -hs *
1.6G    136049aa-d02d-4a.tmp
1.6G    190528d6-ffe7-44.tmp
1.6G    197cb16c-7bd8-49.tmp
91M    241da6c0-ee1f-45.tmp
91M    283785f4-c7a7-4e.tmp
91M    28df1ab1-cafd-4b.tmp
1.6G    2fabe926-dd25-46.tmp
91M    322b5b76-38b6-46.tmp
91M    3892e70d-d10d-41.tmp
1.6G    3d4376f1-748f-4f.tmp
1.6G    45782341-d73b-4c.tmp
91M    4c1f0f27-b592-4c.tmp
1.6G    5803da08-e7fb-4b.tmp
91M    610da03d-dba6-49.tmp
91M    67901f4e-6360-4a.tmp
91M    9e59cf92-ea96-48.tmp
91M    b8ec7a18-6db6-4e.tmp
1.6G    bcd5650b-fc6c-47.tmp
1.6G    c3fceeff-78d4-48.tmp
91M    c44a1bbb-b313-42.tmp
1.6G    c68dc340-e109-4b.tmp
1.6G    cc9752a0-ce66-4d.tmp
91M    d34b2f4a-ab1b-49.tmp
91M    e115d711-0f3f-44.tmp
1.6G    e529f468-8441-4c.tmp
1.6G    f0b979c7-b3b7-48.tmp
91M    f5d8bc81-dd90-45.tmp
@techman83 techman83 added Bug Something is not working as intended Netkan Issues affecting the netkan data Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) labels Oct 9, 2020
@HebaruSan
Copy link
Member

From chat, looks like the cache filled up, so we started failing when we tried to move from /tmp to cache.
We should catch that and delete the .tmp.

@DasSkelett
Copy link
Member

Additionally, to solve the underlying problem of the full cache, we could think about being smarter when cleaning the cache. NetKAN will only ever need the latest version of a mod, so we could remove all the other ones, which could help with frequently updating mods. Right now, this would remove about 12 GB from older HumanStuff releases alone.

However since the cache filenames from netkan don't include version numbers (and even those would be unsafe to parse because of hyphens in identifiers/epochs/version strings), I don't know how we could achivee that yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something is not working as intended Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Netkan Issues affecting the netkan data
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants