Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Liked-by crawler not downloading all posts #187

Closed
thomas694 opened this issue Nov 9, 2021 · 5 comments
Closed

Liked-by crawler not downloading all posts #187

thomas694 opened this issue Nov 9, 2021 · 5 comments
Labels
bug Something isn't working

Comments

@thomas694
Copy link
Contributor

[ed.] I'm running into a[n] [...] issue, specifically with likes, on 2.4.1 (thank you for this recent update!).

I've read your documentation and the issues associated with likes. I've authenticated, clicked the second button which searches for cats, and tumblthree reports being authenticated with my username.

I've added https://www.tumblr.com/likes and my actual blog URL https://www.tumblr.com/liked/by/myblog (which I am slightly reluctant to post here in public, but if it is helpful, I can share it with you) .

I've verified that my likes are publicly visible at that URL using private mode in a second browser.

Only the https://www.tumblr.com/likes URL seems to be able to download at all. No queue of files appears, and it takes a very, very long time for the process to start - like hours. I don't mind this but it freezes after about half (428 of 928) items are downloaded. Oddly, I have around 2500 likes but this is not reflected in the number of downloads.

I've opened tumblthree.log (attached) and it seems to be primarily complaining about the clipboard, but I am not really qualified to intepret what's happening.

I'm not clear on how to continue troubleshooting, and hope I am not missing something simple here. Thanks again for your work on this app and for reading this.

tumblthree

TumblThree.log

Originally posted by @ViceChief in #168 (comment)

@thomas694
Copy link
Contributor Author

@ViceChief I made a new issue out of your comment.

The log file snippet isn't showing us too much. If you like, you can disable the clipboard monitor by clicking on the clipboard button in the toolbar, they blue highlighting should disappear.

It looks like there might be a problem with the liked-by downloading. I need to have a closer look in the next days.

@thomas694 thomas694 added the bug Something isn't working label Nov 9, 2021
@ViceChief
Copy link

Clipboard checking disabled. I'll be monitoring closely in case there is anything you'd like me to try.

thomas694 added a commit that referenced this issue Nov 13, 2021
- An error while converting the downloaded page with posts could interrupt the crawl process.
- Now trying to parse the posts even no successful conversion was done.
@thomas694
Copy link
Contributor Author

The issue has been fixed and closed. You can still comment. Feel free to ask for reopening the issue if needed.

@ViceChief
Copy link

Worked with version 2.4.2 . For what it's worth, I noticed "Downloaded files" and "number of downloads" on the UI update sporadically, and the process appears to take minutes to begin, however, once it started, all seemed well. Obviously time-to-start doesn't really matter. Thanks for the quick fix.

@ViceChief
Copy link

Also for the record, it was the /liked/by/ URL that worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants