Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Versions still appear in the Files view after running php occ files_versions:cleanup #39046

Closed
5 of 8 tasks
Tracked by #38431
elpraga opened this issue Jun 28, 2023 · 15 comments · Fixed by #43263
Closed
5 of 8 tasks
Tracked by #38431
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug feature: versions

Comments

@elpraga
Copy link

elpraga commented Jun 28, 2023

⚠️ This issue respects the following points: ⚠️

Bug description

I may have run into a problem.

By mistake, I have manually deleted old versions directly in the Nextcloud data directory directly on the server.

Then, after reading about the correct solution, I ran php occ files_versions:cleanup myuser.

I was under the assumption that after this step, old versions will no longer appear in the Files app in the web interface.

They do appear, and obviously, they are inaccessible.

Is this the expected behavior and can this situation on my server be fixed?

Thank you for your reply and advice!

P.S.
I've marked the issue as a bug because I believe that after running php occ files_versions:cleanup myuser, the old versions should not appear regardless of my previous error.

I did run occ files:scan myuser .

Steps to reproduce

  1. run php occ files_versions:cleanup myuser
  2. old versions are still visible

Expected behavior

old versions are no longer visible after running php occ files_versions:cleanup myuser (regardless if they had physically existed on disk or not)

Installation method

None

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "filelocking.enabled": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "elpraga.no-ip.biz",
            "cloud.tomaskaluza.net",
            "192.168.1.146",
            "raspberry",
            "127.0.0.1",
            "192.168.0.10"
        ],
        "enable_previews": true,
        "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
        "preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir \/tmp ",
        "preview_max_x": 1920,
        "preview_max_y": 1080,
        "enabledPreviewProviders": {
            "0": "OC\\Preview\\PNG",
            "1": "OC\\Preview\\JPEG",
            "2": "OC\\Preview\\GIF",
            "3": "OC\\Preview\\Image",
            "5": "OC\\Preview\\Epub",
            "6": "OC\\Preview\\OpenDocument",
            "7": "OC\\Preview\\Font",
            "11": "OC\\Preview\\Illustrator",
            "12": "OC\\Preview\\Postscript",
            "13": "OC\\Preview\\Photoshop",
            "14": "OC\\Preview\\TIFF",
            "15": "OC\\Preview\\MarkDown",
            "16": "OC\\Preview\\Movie",
            "17": "OC\\Preview\\Font",
            "18": "OC\\Preview\\PDF"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "version": "27.0.0.8",
        "installed": true,
        "default_phone_region": "ES",
        "auth.bruteforce.protection.enabled": true,
        "appstoreenabled": true,
        "apps_paths": [
            {
                "path": "\/var\/www\/cloud.tomaskaluza.net\/apps",
                "url": "\/apps",
                "writable": true
            }
        ],
        "forcessl": true,
        "theme": "",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "forceSSLforSubdomains": true,
        "appstore.experimental.enabled": true,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 0,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "minimum.supported.desktop.version": "2.0.0",
        "quota_include_external_storage": false,
        "overwrite.cli.url": "https:\/\/cloud.tomaskaluza.net",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "htaccess.RewriteBase": "\/",
        "dbtype": "mysql",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "beta",
        "mysql.utf8mb4": true,
        "app_install_overwrite": [
            "files_texteditor",
            "richdocumentscode_arm64",
            "facerecognition",
            "health",
            "breezedark"
        ],
        "versions_retention_obligation": "1 1",
        "has_rebuilt_cache": true,
        "memories.exiftool_no_local": true,
        "memories.ffmpeg_path": "\/usr\/bin\/ffmpeg",
        "memories.ffprobe_path": "\/usr\/bin\/ffprobe",
        "memories.no_transcode": false,
        "memories.transcoder": "\/var\/www\/cloud.tomaskaluza.net\/apps\/memories\/exiftool-bin\/go-vod-aarch64",
        "memories.qsv": false,
        "memories.vod.path": "\/var\/www\/cloud.tomaskaluza.net\/apps\/memories\/exiftool-bin\/go-vod-aarch64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - analytics: 4.9.4
  - bookmarks: 13.0.1
  - breezedark: 26.0.0
  - bruteforcesettings: 2.7.0
  - calendar: 4.4.2
  - circles: 27.0.0
  - cloud_federation_api: 1.10.0
  - collectives: 2.5.0
  - comments: 1.17.0
  - contacts: 5.3.2
  - contactsinteraction: 1.8.0
  - cookbook: 0.10.2
  - cospend: 1.5.10
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.10.0
  - event_update_notification: 2.2.0
  - external: 5.2.0
  - facerecognition: 0.9.20
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_automatedtagging: 1.17.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_texteditor: 2.15.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - forms: 3.3.1
  - gpxpod: 5.0.12
  - groupfolders: 15.0.0
  - health: 1.6.2
  - imageconverter: 1.3.5
  - impersonate: 1.14.0
  - integration_deepl: 1.1.0
  - integration_giphy: 1.0.6
  - integration_google: 2.0.2
  - integration_openai: 1.0.10
  - integration_replicate: 1.0.6
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - mail: 3.2.2
  - memories: 5.2.0-beta.1
  - music: 1.8.4
  - news: 22.0.0-beta2
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.0
  - notifications: 2.15.0
  - oauth2: 1.15.0
  - password_policy: 1.17.0
  - passwords: 2023.7.30-build4960
  - photos: 2.3.0
  - polls: 5.1.0
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recognize: 4.2.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - richdocuments: 8.1.0
  - richdocumentscode_arm64: 23.5.5
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - social: 0.6.1
  - spreed: 17.0.1
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - tables: 0.5.1
  - tasks: 0.15.0
  - text: 3.8.0
  - text_templates: 1.0.3
  - theming: 2.2.0
  - theming_customcss: 1.14.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_nextcloud_notification: 3.7.0
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflow_pdf_converter: 1.12.0
  - workflow_script: 1.12.0
  - workflowengine: 2.9.0
Disabled:
  - apporder: 0.15.0 (installed 0.15.0)
  - audioplayer: 3.4.0 (installed 3.4.0)
  - audioplayer_editor: 0.3.0 (installed 0.3.0)
  - camerarawpreviews: 0.8.1 (installed 0.8.1)
  - checksum: 1.2.2 (installed 1.2.2)
  - cms_pico: 1.1.0-beta.1 (installed 1.0.19)
  - data_request: 1.14.0 (installed 1.14.0)
  - drawio: 2.1.1 (installed 2.1.1)
  - duplicatefinder: 0.0.15 (installed 0.0.15)
  - encryption: 2.15.0
  - epubreader: 1.4.7 (installed 1.4.7)
  - extract: 1.3.5 (installed 1.3.5)
  - files_3d: 0.5.0 (installed 0.5.0)
  - files_markdown: 2.4.1 (installed 2.4.1)
  - files_photospheres: 1.26.4 (installed 1.26.4)
  - files_readmemd: 2.0.0 (installed 2.0.0)
  - files_retention: 1.16.0 (installed 1.16.0)
  - gestion: 2.3.3 (installed 2.3.3)
  - google_synchronization: 1.0.9 (installed 1.0.9)
  - gpxedit: 0.0.14 (installed 0.0.14)
  - group_everyone: 0.1.13 (installed 0.1.13)
  - issuetemplate: 0.7.0 (installed 0.7.0)
  - maps: 1.0.2 (installed 1.0.2)
  - metadata: 0.18.0 (installed 0.18.0)
  - nextbackup: 21.1.0 (installed 21.1.0)
  - ocsms: 2.2.0 (installed 2.2.0)
  - onlyoffice: 8.1.0 (installed 8.1.0)
  - phonetrack: 0.7.6 (installed 0.7.6)
  - quicknotes: 0.8.10 (installed 0.8.10)
  - radio: 1.0.3 (installed 1.0.3)
  - scanner: 0.2.3 (installed 0.2.3)
  - sharerenamer: 3.2.0 (installed 3.2.0)
  - talk_simple_poll: 1.3.1 (installed 1.3.1)
  - timemanager: 0.3.7 (installed 0.3.7)
  - timetracker: 0.0.79 (installed 0.0.79)
  - transfer: 0.6.1 (installed 0.6.1)
  - user_ldap: 1.17.0
  - user_usage_report: 1.11.0 (installed 1.11.0)
  - video_converter: 1.0.5 (installed 1.0.5)
  - weather: 1.7.7 (installed 1.7.7)
  - whiteboard: 0.0.3 (installed 0.0.3)

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

@elpraga elpraga added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jun 28, 2023
@elpraga

This comment was marked as resolved.

@szaimen
Copy link
Contributor

szaimen commented Jul 10, 2023

cc @artonge

@artonge

This comment was marked as resolved.

@elpraga

This comment was marked as spam.

@joshtrichards
Copy link
Member

Can you check your Nextcloud log and pull the stack trace or error returned when you attempt to access these bogus versions?

@artonge
Copy link
Contributor

artonge commented Sep 21, 2023

Can you run the occ file:scan command and see if they are still visible ?

@Nypheena
Copy link

I have this issue too and file:scan and file:cleanup doesn't fix it, I use a hetzner storage share and they saw after I deleted the files they were still showing in the database I also found this #37461 issue and I would suspect its related although for this its in reverse, it is not removing it in the database it needs to once the files are deleted through OCC or expire through the retention setting but it works if i do a manual delete in the webui

@Anonym777
Copy link

Anonym777 commented Nov 12, 2023

Hello,
I have the same issue. I deleted the old versions in the data directory directly (because I did not need them any longer) after a migration. And now the old versions still appear in the UI. The UI also becomes laggy when trying to access the Versions tab for such a file. I was able to deleted the old/inaccessible versions for this file manually from the UI and the UI stopped lagging.
old_versions

Seems that the old versions are still referenced somewhere. The database I imagine. Is there a way to clean these references safely from the database?

@ruedigerkupper
Copy link

Our installation appears to be suffering from the same ptoblem. Not caused by manual deletion, but possibly problems with external storage. The effect is as described though, and I would be interested in how to remove these stale database entries referring to version files that do not physically exist.

@artonge
Copy link
Contributor

artonge commented Jan 25, 2024

I was not able to reproduce. Is the issue still occurring?
Could one of you list some reproduction steps to get into such state?

@Nypheena
Copy link

Nypheena commented Jan 25, 2024

@artonge

I was not able to reproduce. Is the issue still occurring? Could one of you list some reproduction steps to get into such state?

What version you on? Because I do not believe anything has changed with v27

As for reproducing, all you gotta do is upload a file twice so it gets versioned then run the occ command to clear out all versions and you’ll notice while the files on disk is removed the database still have them so they’ll show in the versions list on the uploaded file

for me this also applies with retention settings when auto cleared it also doesn’t clear from its database so it always shows all versions that ever existed just the version file itself no longer exists on disk

EDIT: I just tested on my hetzner storage share which runs 27.1.5 and the issue remains, the versions all appear on the list for the file but they are deleted on disk, database is not cleared of them

@justsomescripts
Copy link

I can reproduce what @Zairhn did on Nextcloud v28.0.1.1

@artonge
Copy link
Contributor

artonge commented Jan 30, 2024

@justsomescripts @Zairhn I was not able to reproduce. Can you check if there is anything related in your Nextcloud server logs? You can check the log file just after running the command.
Can you confirm that the command is occ versions:expire and not occ files_versions:cleanup?

@Nypheena
Copy link

@artonge

@justsomescripts @Zairhn I was not able to reproduce. Can you check if there is anything related in your Nextcloud server logs? You can check the log file just after running the command. Can you confirm that the command is occ versions:expire and not occ files_versions:cleanup?

the command is not versions:expire it is infact cleanup as the title of this bug report states, but I have had the same issue with versions:expire, I also don't have log access on hetzner storage share

@artonge
Copy link
Contributor

artonge commented Feb 1, 2024

Hello, could one of you test this PR: #43263
It adds a step to flush the files_versions table before deleting the files_versions folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 27-feedback bug feature: versions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants