Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of requests library could lead to partial file downloads without notice #10

Closed
puigru opened this issue Feb 26, 2023 · 1 comment · Fixed by #11
Closed

Use of requests library could lead to partial file downloads without notice #10

puigru opened this issue Feb 26, 2023 · 1 comment · Fixed by #11

Comments

@puigru
Copy link
Contributor

puigru commented Feb 26, 2023

I've recently begun using this tool to build my music library. However, I noticed something odd after running it a second time, as it re-downloaded a couple of files because their size did not match up with what it should have been. Yet, there had been no errors on the previous run. I quickly realized this could be another instance of a well-known requests shortcoming.

Unfortunately, unlike other HTTP tools, requests does not check the response's Content-Length header to ensure it has been fully downloaded, leaving the burden on its user to do so. Hence, it's possible to end up with partial downloads by using bandcamp-downloader, as it does not perform this check. Although, it will attempt to re-download these files on subsequent runs.

Read:
https://blog.petrzemek.net/2018/04/22/on-incomplete-http-reads-and-the-requests-library-in-python/
psf/requests#4956 (and issues linked within)

@easlice
Copy link
Owner

easlice commented Feb 26, 2023

I thought I had seen this before, but I wasn't sure as to the cause. I appreciate the effort and the input of someone who knows these libs better than I do! Also, I really appreciate your thorough explanations and submitting patches to boot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants