-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated featurizers #4935
Merged
Merged
Updated featurizers #4935
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@tabergma you can change the model compatibility version in rasa/constants.py |
tmbo
approved these changes
Dec 16, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great work!
I've added some style suggestions but from my perspective this is ready to go
rasa/nlu/featurizers/sparse_featurizer/count_vectors_featurizer.py
Outdated
Show resolved
Hide resolved
Co-Authored-By: Tom Bocklisch <tom@rasa.com>
…r.py Co-Authored-By: Tom Bocklisch <tom@rasa.com>
…r.py Co-Authored-By: Tom Bocklisch <tom@rasa.com>
tabergma
force-pushed
the
updated-featurizers
branch
from
December 16, 2019 14:56
e4328b8
to
ad30827
Compare
tabergma
force-pushed
the
updated-featurizers
branch
from
December 16, 2019 16:43
7f9a1c6
to
b83ee6f
Compare
tmbo
reviewed
Dec 16, 2019
woop woop 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes:
use_cls_token
to all tokenizers. If it is set toTrue
, the token__CLS__
will be added to the end of the list of tokens.return_sequence
to all featurizers. By default all featurizers return a matrix of size(1 x feature-dimension). If the option
return_sequence
is set toTrue
, the corresponding featurizer will return a matrix of size (token-length x feature-dimension).NGramFeaturizer
. Please useCountVectorsFeaturizer
instead.CRFEntityExtractor
usetext_dense_features
instead ofner_features
. Iftext_dense_features
are present in the feature set, theCRFEntityExtractor
will automatically make use of them. Just make sure to add a dense featurizer in front of theCRFEntityExtractor
in your pipeline and set the flagreturn_sequence
toTrue
for that featurizer.closes #4957
part of https://github.com/RasaHQ/research/issues/54
Status (please check what you already did):
black
(please check Readme for instructions)