Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Random sample on stream #1444

Merged
merged 10 commits into from
Aug 24, 2017
Merged

Random sample on stream #1444

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6b4c3a8
Create local random generator for sample_text & add lenght
vlejd Jun 16, 2017
ab3f517
Fix typos in textcorpus.py
vlejd Jun 17, 2017
c0e375b
Fix typo in yields (yeilds)
vlejd Jun 18, 2017
0015898
Fix typos and clarify exeception in sample_texts
vlejd Jun 18, 2017
5395aba
Merge branch 'develop' into 308_random_sample_on_stream
vlejd Jun 22, 2017
8e715f6
Improve error messages
vlejd Jun 22, 2017
dcb0fe7
Merge branch 'develop' into 308_random_sample_on_stream
vlejd Jun 22, 2017
e11a028
Merge branch 'develop' into 308_random_sample_on_stream
menshikh-iv Jul 5, 2017
610a3f7
remove seen_documents
menshikh-iv Aug 10, 2017
b6b0927
cosmetic changes
menshikh-iv Aug 24, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions gensim/corpora/textcorpus.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,7 @@ def sample_texts(self, n, seed=None, length=None):
if not 0 <= n:
raise ValueError("Negative sample size n {0:d}.".format(n))

seen_documents = 0
for i, sample in enumerate(self.get_texts()):
seen_documents = i + 1

if i == length:
break

Expand All @@ -305,7 +302,7 @@ def sample_texts(self, n, seed=None, length=None):
# This means that length was set to be greater than number of items in corpus
# and we were not able to sample enough documents before the stream ended.
raise ValueError(
"length {0:d} greater than number of documents in corpus {1:d}".format(length, seen_documents))
"length {0:d} greater than number of documents in corpus {1:d}".format(length, i + 1))
Copy link
Owner

@piskvorky piskvorky Aug 24, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail with an exception if there are no texts (i will be undefined).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i undefined if only self.getstream() is empty


def __len__(self):
if self.length is None:
Expand Down