Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache special drive items until space root changes #6189

Merged
merged 8 commits into from
May 3, 2023

Conversation

butonic
Copy link
Member

@butonic butonic commented Apr 28, 2023

When listing drives we now cach special drive items until the space root changes again or the TTL is reached.

brings back the cache that was dropped in #3141

@update-docs
Copy link

update-docs bot commented Apr 28, 2023

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@ownclouders
Copy link
Contributor

ownclouders commented Apr 28, 2023

💥 Acceptance test localApiTests-apiSpaces-ocis failed. Further test are cancelled...

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic force-pushed the cache-special-drive-items branch from 0d6ea63 to ca638dd Compare May 2, 2023 09:44
@butonic
Copy link
Member Author

butonic commented May 2, 2023

tar: Unexpected EOF in archive in https://drone.owncloud.com/owncloud/ocis/21953/67/1

restarted the build ...

@butonic
Copy link
Member Author

butonic commented May 2, 2023

hm, dhe cache works correctly, but when determining which special items to fetch the space opaque never contains a readme or image key because in the failing tests the file is uploaded directly, but the code in reva and the getSpecialDriveItems implementation work on the assumption that the space node contains name properties for the two special folders ...

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic force-pushed the cache-special-drive-items branch from c3881bd to f2b86d8 Compare May 2, 2023 15:13
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic
Copy link
Member Author

butonic commented May 2, 2023

It seems the size is wrong ... maybe because the cache for other users was not invalidated? but the gateway should not have a stat cache ... a problem for future me ...

  Scenario Outline: user set new readme file as description of the space via the graph API                                                                           # /drone/src/tests/acceptance/features/apiSpaces/changeSpaces.feature:440
    Given user "Alice" has created a folder ".space" in space "Project Jupiter"                                                                                      # SpacesContext::theUserHasCreateAFolderUsingTheGraphApi()
    And user "Alice" has uploaded a file inside space "Project Jupiter" with content "space description" to ".space/readme.md"                                       # SpacesContext::userHasUploadedFile()
    And user "Alice" has set the file ".space/readme.md" as a description in a special section of the "Project Jupiter" space                                        # SpacesContext::userHasUpdatedSpaceSpecialSection()
    When user "<user>" uploads a file inside space "Project Jupiter" owned by the user "Alice" with content "new content" to ".space/readme.md" using the WebDAV API # SpacesContext::theUserUploadsAFileToAnotherOwnerSpace()
    Then the HTTP status code should be "<code>"                                                                                                                     # FeatureContext::thenTheHTTPStatusCodeShouldBe()
    And for user "<user>" the content of the file ".space/readme.md" of the space "Project Jupiter" should be "<expectedContent>"                                    # SpacesContext::checkFileContent()
    And for user "<user>" the JSON response should contain space called "Project Jupiter" owned by "Alice" with description file ".space/readme.md" and match        # SpacesContext::forUserTheJSONDataOfTheResponseShouldMatch()
      """
      {
        "type": "object",
        "required": [
          "name",
          "special"
        ],
        "properties": {
          "name": {
            "type": "string",
            "enum": ["Project Jupiter"]
          },
          "special": {
            "type": "array",
            "items": [
              {
                "type": "object",
                "required": [
                  "size",
                  "name",
                  "specialFolder",
                  "file",
                  "id",
                  "eTag"
                ],
                "properties": {
                  "size": {
                    "type": "number",
                    "enum": [<expectedSize>]
                  },
                  "name": {
                    "type": "string",
                    "enum": ["readme.md"]
                  },
                  "specialFolder": {
                    "type": "object",
                    "required": [
                      "name"
                    ],
                    "properties": {
                      "name": {
                        "type": "string",
                        "enum": ["readme"]
                      }
                    }
                  },
                  "file": {
                    "type": "object",
                    "required": [
                      "mimeType"
                    ],
                    "properties": {
                      "type": "string",
                      "enum": ["text/markdown"]
                    }
                  },
                  "id": {
                    "type": "string",
                    "enum": ["%file_id%"]
                  },
                  "tag": {
                    "type": "string",
                    "enum": ["%eTag%"]
                  }
                }
              }
            ]
          }
        }
      }
      """

    Examples:
      | user  | code | expectedSize | expectedContent   |
      | Alice | 204  | 11           | new content       |
      | Brian | 204  | 11           | new content       |
        Failed step: And for user "Brian" the JSON response should contain space called "Project Jupiter" owned by "Alice" with description file ".space/readme.md" and match
        Failed asserting that stdClass Object &0000000073876f1400000000473029be (
            'driveAlias' => 'project/project-jupiter'
            'driveType' => 'project'
            'id' => 'dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959'
            'lastModifiedDateTime' => '2023-05-02T20:11:44.501120002Z'
            'name' => 'Project Jupiter'
            'owner' => stdClass Object &0000000073876f1b00000000473029be (
                'user' => stdClass Object &0000000073876f1a00000000473029be (
                    'displayName' => ''
                    'id' => '25fcf950-e88d-4191-b820-23ae6b060959'
                )
            )
            'quota' => stdClass Object &0000000073876f1300000000473029be (
                'remaining' => 9
                'state' => 'normal'
                'total' => 20
                'used' => 11
            )
            'root' => stdClass Object &0000000073876f1500000000473029be (
                'eTag' => '"5af12c1021fffe5d120f294dc21eefcc"'
                'id' => 'dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959'
                'permissions' => Array &0 (
                    0 => stdClass Object &0000000073876f1800000000473029be (
                        'grantedToIdentities' => Array &1 (
                            0 => stdClass Object &0000000073876f2c00000000473029be (
                                'user' => stdClass Object &0000000073876f2d00000000473029be (
                                    'displayName' => 'Alice Hansen'
                                    'id' => '4c783806-b931-4c4a-b1b6-c862d9f3f3a9'
                                )
                            )
                        )
                        'roles' => Array &2 (
                            0 => 'manager'
                        )
                    )
                    1 => stdClass Object &0000000073876f1c00000000473029be (
                        'grantedToIdentities' => Array &3 (
                            0 => stdClass Object &0000000073876f3400000000473029be (
                                'user' => stdClass Object &0000000073876f2400000000473029be (
                                    'displayName' => 'Regular User'
                                    'id' => '678a2e7d-ba11-4272-a0a7-cbeb5206ef3c'
                                )
                            )
                        )
                        'roles' => Array &4 (
                            0 => 'viewer'
                        )
                    )
                    2 => stdClass Object &0000000073876f0c00000000473029be (
                        'grantedToIdentities' => Array &5 (
                            0 => stdClass Object &0000000073876f1000000000473029be (
                                'user' => stdClass Object &0000000073876f2e00000000473029be (
                                    'displayName' => 'Brian Murphy'
                                    'id' => 'b8aafa81-8fb7-4c84-9bea-2520a5348b96'
                                )
                            )
                        )
                        'roles' => Array &6 (
                            0 => 'editor'
                        )
                    )
                )
                'webDavUrl' => 'https://ocis-server:9200/dav/spaces/dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959'
            )
            'special' => Array &7 (
                0 => stdClass Object &0000000073876f0d00000000473029be (
                    'eTag' => '"dfc88400759b218e57846e330a5c330c"'
                    'file' => stdClass Object &0000000073876f1900000000473029be (
                        'mimeType' => 'text/markdown'
                    )
                    'id' => 'dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959!6b7e7add-b63a-4843-961e-e3802cb80d54'
                    'lastModifiedDateTime' => '2023-05-02T20:11:44.125259354Z'
                    'name' => 'readme.md'
                    'size' => 17
                    'specialFolder' => stdClass Object &0000000073876f2300000000473029be (
                        'name' => 'readme'
                    )
                    'webDavUrl' => 'https://ocis-server:9200/dav/spaces/dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959%2125fcf950-e88d-4191-b820-23ae6b060959/.space/readme.md'
                )
            )
            'webUrl' => 'https://ocis-server:9200/f/dabffcc8-9620-4685-907e-3926beefeb08$25fcf950-e88d-4191-b820-23ae6b060959'
        ) matches JSON schema.
        [special[0].size] Does not have a value in the enumeration [11]
      | Bob   | 403  | 17           | space description |

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic
Copy link
Member Author

butonic commented May 3, 2023

when updating space metadata we do not update the mtime (because updating xatrs only changes the chtime and when moving to messagepack metadata we kept that). This prevents us from using the mtime / etag of the space to invalidate the stat cache for the image: first the special file is uploaded (image or readme), the mtime propagates to the root. then the space metadata is changed, not causing an mtime change. the code checks if space mtime and cached mtime are the same ... hmmm

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@sonarcloud
Copy link

sonarcloud bot commented May 3, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

79.2% 79.2% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@micbar micbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@micbar micbar merged commit b354e79 into master May 3, 2023
@delete-merged-branch delete-merged-branch bot deleted the cache-special-drive-items branch May 3, 2023 16:16
ownclouders pushed a commit that referenced this pull request May 3, 2023
cache special drive items until space root changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants