Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix os.rename from gensim.downloader when src and dst on different partitions #1733

Merged
merged 14 commits into from
Nov 23, 2017
Merged

Fix os.rename from gensim.downloader when src and dst on different partitions #1733

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3cc34ff
Fix typo
anotherbugmaster Sep 30, 2017
cde582e
Make `save_corpus` private
anotherbugmaster Oct 2, 2017
34cccfe
Annotate `bleicorpus.py`
anotherbugmaster Oct 2, 2017
34bd9ef
Make __save_corpus weakly private
anotherbugmaster Oct 2, 2017
3b55037
Fix _save_corpus[2]
anotherbugmaster Oct 3, 2017
04c2e61
Document bleicorpus in Numpy style
anotherbugmaster Oct 24, 2017
6c6d118
Fix tests on Arch Linux
anotherbugmaster Nov 21, 2017
9ef0206
Revert "Document bleicorpus in Numpy style"
anotherbugmaster Nov 23, 2017
7f103f9
Revert "Fix _save_corpus[2]"
anotherbugmaster Nov 23, 2017
566430a
Revert "Make __save_corpus weakly private"
anotherbugmaster Nov 23, 2017
a583f40
Revert "Annotate `bleicorpus.py`"
anotherbugmaster Nov 23, 2017
026b1f7
Revert "Make `save_corpus` private"
anotherbugmaster Nov 23, 2017
df9903a
Revert "Fix typo"
anotherbugmaster Nov 23, 2017
eaf8bd2
Handle the multiple partitions case
anotherbugmaster Nov 23, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions gensim/downloader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ def _download(name):
"""
url_load_file = "{base}/{fname}/__init__.py".format(base=DOWNLOAD_BASE_URL, fname=name)
data_folder_dir = os.path.join(base_dir, name)
data_folder_dir_tmp = data_folder_dir + '_tmp'
tmp_dir = tempfile.mkdtemp()
init_path = os.path.join(tmp_dir, "__init__.py")
urllib.urlretrieve(url_load_file, init_path)
Expand Down Expand Up @@ -310,7 +311,6 @@ def _download(name):
with open(part_path, "rb") as rfp:
shutil.copyfileobj(rfp, wfp)
os.remove(part_path)
os.rename(tmp_dir, data_folder_dir)
else:
url_data = "{base}/{fname}/{fname}.gz".format(base=DOWNLOAD_BASE_URL, fname=name)
fname = "{fname}.gz".format(fname=name)
Expand All @@ -323,7 +323,12 @@ def _download(name):
else:
shutil.rmtree(tmp_dir)
raise Exception("Checksum comparison failed, try again")
os.rename(tmp_dir, data_folder_dir)

if os.path.exists(data_folder_dir_tmp):
os.remove(data_folder_dir_tmp)

shutil.move(tmp_dir, data_folder_dir_tmp)
os.rename(data_folder_dir_tmp, data_folder_dir)


def _get_filename(name):
Expand Down