-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve sync speed and updated dep. versions #2429
Conversation
a184bbe
to
1fa2ed1
Compare
Just wondering if this crate could be used https://crates.io/crates/tokio_cron_scheduler instead of a not so maintained version of the current job_scheduler. It is based on job_scheduler, and seem actively maintained for now at least. |
@manofthepeace I'm not happy with the And, when i did try to integrate that crate, i bumbed into some issues with passing on some variables because of how async works. Also, we think it isn't that bad to have the scheduled tasks to run in there own thread to not bother the application thread it self with this. As you can see i already forked it my self and updated the dependencies where needed. I might tend to publish the crate since the original author of the crate doesn't seem to be working on it at all anymore. |
1fa2ed1
to
ad4d199
Compare
Small update. I have fixed all the full-sync code to use the the I'm still checking for other optimizations to see if I can add those. |
ad4d199
to
ebf92cb
Compare
Did some final checking and this looks like merge-ready. There are also some optimizations which could be done with web-sockets, but also that need some more work and probably better for an other PR. |
Improved sync speed by resolving the N+1 query issues. Solves dani-garcia#1402 and Solves dani-garcia#1453 With this change there is just one query done to retreive all the important data, and matching is done in-code/memory. With a very large database the sync time went down about 3 times. Also updated misc crates and Github Actions versions.
ebf92cb
to
3ca8502
Compare
Thanks for your work @BlackDex! |
Just wanted to say that this is the greatest thing. Loading the home list went down from 4-8s to <0.5s. Best optimisation since the bread slicer! |
Improved sync speed by resolving the N+1 query issues.
Solves #1402 and Solves #1453
With this change there is just one query done to retreive all the important data, and matching is done in-code/memory.
With a very large database the sync time went down about 3 times.
Also updated misc crates and Github Actions versions.