-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA]: Remove redundant bert hash files #850
Labels
feature request
New feature or request
Comments
Similarly we have two copies of
|
3 tasks
rapids-bot bot
pushed a commit
that referenced
this issue
Apr 28, 2023
This PR creates at least one test for each example containing custom stages. This PR currently only covers those examples which do not require additional packages. Part of #849. * Moves the bert vocabulary files to `morpheus/data` dir, no longer requiring them to be fetched from LFS and making them available to unittests. * Fixes type hints and remove a redundant method in `examples/log_parsing/inference.py` * Remove redundant copies of `bert-base-cased-hash.txt` and `bert-base-uncased-hash.txt` files, replacing them with symlinks to the files in the morpheus/data` dir fixes #850 * Explicitly set `encoding='UTF-8'` in `examples/log_parsing/postprocessing.py` as a work-around for issue #859 * Add `py::kw_only` to Python bindings for `TensorMemory` and sublasses to ensure parity with Python impls. * Set `repr=False` for the `tensors` field of `TensorMemory` avoids bug when printing due to the fact that we assign the value to `self._tensors` * Seed cupy's random number generator in `manual_seed` method. * Fix usage of `reload_modules` fixture, requesting a reload of multiple modules should be done with `@pytest.mark.reload_modules([mod1, mod2])` not calling `reload_modules` twice. * New test data in `tests/tests_data/log_parsing` is based upon the first 5 rows of data from `models/datasets/validation-data/log-parsing-validation-data-input.csv` Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #885
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is this a new feature, an improvement, or a change to existing functionality?
Change
How would you describe the priority of this feature request
Low (would be nice)
Please provide a clear description of problem this feature solves
Looks like we have a couple of different copies of these, not sure if git-lfs is smart enough to prevent redundant copies of these, but even if it is, we have these also stored in the
morpheus/data
dir as regular git files.Describe your ideal solution
delete
Describe any alternatives you have considered
No response
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: