Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.3.1 #79

Merged
merged 13 commits into from
Oct 29, 2021
Merged

v0.3.1 #79

merged 13 commits into from
Oct 29, 2021

Conversation

thammegowda
Copy link
Owner

@thammegowda thammegowda commented Oct 28, 2021 •
edited
Loading

  • Add support for recipes; list-recipe get-recipe subcommands added
  • add support for viewing stats of dataset; words, chars, segs
  • FIX URL for UN dev and test sets (source was updated so we updated too)
  • Multilingual experiment support; ISO 639-3 code mul implies multilingual; e.g. mul-eng or eng-mul
  • --dev accepts multiple datasets, and merges them (useful for multilingual experiments)
  • tar files are extracted before reading (performance improvements) Closes Reading from tarfiles without extracting them is slow  #78
  • setup.py: version and descriptions accessed via regex

@thammegowda thammegowda changed the title v0.3.1 - WIP v0.3.1 Oct 29, 2021
@thammegowda thammegowda merged commit 4380a10 into master Oct 29, 2021
@thammegowda thammegowda deleted the develop branch October 29, 2021 01:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Reading from tarfiles without extracting them is slow
1 participant
@thammegowda