Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preprocessing Script Question #77

Open
hdeval1 opened this issue Jun 16, 2022 · 0 comments
Open

Preprocessing Script Question #77

hdeval1 opened this issue Jun 16, 2022 · 0 comments

Comments

@hdeval1
Copy link

hdeval1 commented Jun 16, 2022

I realized the preprocessing scripts in the OPUS-MT-Train library did not match the ones being published in the OPUS models repository. I am thinking the preprocess scripts in the training library (scripts/) are outdated, because when i used those to train my own model, i ran into issues. I updated those to the attached script (one I pulled from a model in the repo) and things went smoothly. I just want to make sure I am correct in replacing it. This is for building a SPM model, so I replaced scripts/preprocess-spm.sh with the attached file.
preprocess.sh.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant