Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement optimized getDirectoryContent for DAV #38945

Merged
merged 1 commit into from
Jun 22, 2023

Conversation

icewind1991
Copy link
Member

  • implement getDirectoryContent with a single propfind and rework opendir to reuse it
  • implement getMetaData and rework other metadata methods to reuse it
  • create shared logic to parse propfind response into file metadata

This both ensures that we only need to make a single propfind request to access a folder, and makes it fill the statcache so that any other metadata operations can reuse the cached info.
(Previously opendir didn't fill the cache because it didn't request the same properties)

@icewind1991 icewind1991 added the 3. to review Waiting for reviews label Jun 22, 2023
@icewind1991 icewind1991 added this to the Nextcloud 28 milestone Jun 22, 2023
@icewind1991 icewind1991 force-pushed the dav-meta-directory-content branch from e91fcfa to 0b688c5 Compare June 22, 2023 13:01
@icewind1991 icewind1991 marked this pull request as ready for review June 22, 2023 13:03
Copy link
Member

@juliusknorr juliusknorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, also does the trick and is indeed a bit cleaner 👍

Thanks @icewind1991

@juliusknorr juliusknorr requested review from come-nc and blizzz June 22, 2023 13:32
@icewind1991 icewind1991 added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Jun 22, 2023
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991 icewind1991 force-pushed the dav-meta-directory-content branch from 0b688c5 to 279822c Compare June 22, 2023 14:34
@icewind1991 icewind1991 added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Jun 22, 2023
@icewind1991 icewind1991 merged commit 7fa941e into master Jun 22, 2023
@icewind1991 icewind1991 deleted the dav-meta-directory-content branch June 22, 2023 16:14
@icewind1991
Copy link
Member Author

/backport to stable27

@icewind1991
Copy link
Member Author

/backport to stable26

@backportbot-nextcloud
Copy link

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27

# Create the new backport branch
git checkout -b fix/foo-stable27

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts. Resolve them.
git cherry-pick abc123

# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable27

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

@backportbot-nextcloud
Copy link

The backport to stable26 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable26
git pull origin stable26

# Create the new backport branch
git checkout -b fix/foo-stable26

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts. Resolve them.
git cherry-pick abc123

# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable26

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

@solracsf
Copy link
Member

/backport to stable27

@solracsf
Copy link
Member

/backport to stable26

@Alexandero89
Copy link

Alexandero89 commented Jun 23, 2023

@icewind1991 and @solracsf thanks for your effort, it seems to boost the performance!

I noticed we maybe should add
$file = $this->cleanPath($file);
in between 925 and 926. So it should look like this:

$file = urldecode($file);
$file = substr($file, strlen($this->root));
$file = $this->cleanPath($file);
$this->statCache->set($file, $response);
yield $this->getMetaFromPropfind($file, $response);

Otherwise directorypaths are saved incorrectly.

A directory named for example "Backups" is saved to the cache as "Backups/".
But when requesting for existing cache we ask for "Backups".
So directories are never found in the cache.

@backportbot-nextcloud
Copy link

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27

# Create the new backport branch
git checkout -b fix/foo-stable27

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts. Resolve them.
git cherry-pick abc123

# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable27

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

@backportbot-nextcloud
Copy link

The backport to stable26 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable26
git pull origin stable26

# Create the new backport branch
git checkout -b fix/foo-stable26

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts. Resolve them.
git cherry-pick abc123

# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable26

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

@icewind1991

This comment was marked as outdated.

@solracsf

This comment was marked as outdated.

@blizzz
Copy link
Member

blizzz commented Jun 23, 2023

@icewind1991 you linked to this same PR. thinking

#38965

@solracsf
Copy link
Member

/backport to stable27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants