Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query with multiple hashtags very slow #112

Closed
ahheckel opened this issue Apr 11, 2021 · 4 comments · Fixed by #142
Closed

query with multiple hashtags very slow #112

ahheckel opened this issue Apr 11, 2021 · 4 comments · Fixed by #142
Labels
enhancement New feature or request

Comments

@ahheckel
Copy link

I am running linkding in a docker container (following the instructions on this page) on a Raspberry4 under a current RaspianOS 32bit with ~500 bookmarks which works very well so far.
The response is quick when searching for a single hashtag, however it can be very long (up to a minute) when combining multiple hashtags in a query (#tag1 #tag2). This is particularly true when that query yields more than just a few results.
I was wonderung whether the database query can be optimized for speed.
Kind regards,
(PS great project).

@sissbruecker
Copy link
Owner

I tested this a bit and can confirm that query performance degrades when using multiple tags. However in my tests (also on a RPie 4) I would need to add a lot more than 2 tags and even then the query would resolve in a few seconds (which is still too slow IMO, but far better than what you are experiencing).

Specifically for the use-case described here - for 500 bookmarks and 2 tags in the query 1 minute would be extremely bad. I see two possible issues here:

  • You might have a lot of tag assignments in general - if you could check how many rows you have in your bookmarks_bookmark_tags table in the database that might help solving this
  • Or there is just some other reason why your installation performs to poorly. However if regular searches perform well that seems unlikey.

@ulixxe
Copy link
Contributor

ulixxe commented Aug 13, 2021

Hi,
I closed #141 and use this issue from now on to signal that I search for only 1 tag, and I have to restart the docker container after several minutes of 100% CPU utilization.
In my bookmarks_bookmark_tags table, I have about 2.5k rows.

@ulixxe
Copy link
Contributor

ulixxe commented Aug 14, 2021

I found that the search speed is ok if I comment the below lines 98-101 from bookmarks/queries.py. Can some optimization be done on those lines?

for tag_name in query['tag_names']:
    query_set = query_set.filter(
        bookmark__tags__name__iexact=tag_name
    )

@ahheckel
Copy link
Author

ulixxe's suggestion solves the problem in my case, too.

sissbruecker added a commit that referenced this issue Aug 15, 2021
* changed query on tag search for speedup related to issues #112 and #141

* fix tests and only conditionally append tag filter

* add bookmark tags query tests

* reuse bookmark queries for tag queries

* fix tag query test setup

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@googlemail.com>
@sissbruecker sissbruecker added the enhancement New feature or request label Aug 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants