StringNormalizer drops strings when they only contain stop words

[xadupre]

Issue raised in microsoft/onnxruntime#17795 and #998. After investigation, operator StringNormalizer drops strings containing only stopwords. The expected output can be obtained by running the inference for every input independantly.

A possible fix would be to update onnx specifications to not drop strings. Another fix could be to update the converter and loop over rows but that's not very efficient.

[vsbaldeev]

@xadupre Hi! Any updates on this problem ?

As I understand the fast fix would be to process every string independently. RIght ?

[xadupre]

That would work.

[vsbaldeev]

My tests show that performance on InferenceSession run is twice slower without vectorization - 400 rps instead of 800 rps on my server.

[xadupre]

If you are using a loop, it is not really suprising. There is no parallelization even though each row is processed independently.

[vsbaldeev]

If you are using a loop, it is not really suprising. There is no parallelization even though each row is processed independently.

May you suggest some impovement in this case ?

My guess is if I know indices of dropped strings then I can rerun inference session only on them instead loop over all input items.

[vsbaldeev]

Also I want to notice that I don't user stop_words parameter. How exactly set stop_words parameter to empty or None while converting to ONNX model ?

[xadupre]

One issue is StringNormalizer is defined in onnx. To change its behaviour, it has to be changed in onnx and onnxruntime. It is long. onnxruntime-extensions is a project implementing custom kernel without going through the process of validating the definition by onnx mainteners. The best option here is to implement is to implement a custom kernel. I'm hesitating in adding more in onnxruntime or creating a similar projet to host custom kernel dedicated to standard machine learning. But a custom kernel would solve the issue.

[vsbaldeev]

@xadupre What do you mean by custom kernel exactly ?

[xadupre]

A C++ implementation of an operator. Let's assume we define the operator StringNormalizer2. onnxruntime does not know it because it is new so it cannot run the inference unless we give it a piece of code compiled into an assembly. onnxruntime lets you do that. See https://onnxruntime.ai/docs/reference/operators/add-custom-op.html.

[xadupre]

This issue seems to be fixed with scikit-learn==1.5.0. Could you check on your side?