Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

set urllib user-agent #2437

Merged
merged 1 commit into from
Mar 5, 2021
Merged

set urllib user-agent #2437

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pythonforandroid/recipe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import sh
import shutil
import fnmatch
import urllib.request
from urllib.request import urlretrieve
from os import listdir, unlink, environ, mkdir, curdir, walk
from sys import stdout
Expand All @@ -23,6 +24,11 @@
from pythonforandroid.util import load_source as import_recipe


url_opener = urllib.request.build_opener()
url_orig_headers = url_opener.addheaders
urllib.request.install_opener(url_opener)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this change it at global level? Can't we use some kind of context manager or something so it's rolled back after the recipe download is done?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does. But since this is the only use of urllib in p4a and it seems unlikely that a different user agent would matter to any other code anyway, it seemed like the easiest solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the code. Is this better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good to me



class RecipeMeta(type):
def __new__(cls, name, bases, dct):
if name != 'Recipe':
Expand Down Expand Up @@ -206,6 +212,8 @@ def report_hook(index, blksize, size):
seconds = 1
while True:
try:
# jqueryui.com returns a 403 w/ the default user agent
url_opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urlretrieve(url, target, report_hook)
except OSError as e:
attempts += 1
Expand All @@ -215,6 +223,8 @@ def report_hook(index, blksize, size):
time.sleep(seconds)
seconds *= 2
continue
finally:
url_opener.addheaders = url_orig_headers
break
return target
elif parsed_url.scheme in ('git', 'git+file', 'git+ssh', 'git+http', 'git+https'):
Expand Down