[Good First Issue][TF FE]: Support complex tensors for PadV2 operations #22955
Comments
rkazants
added
no_stale
rkazants
added
gsoc-prerequisite-task
|
.take |
|
Thank you for looking into this issue! Please let us know if you have any questions or require any help. |
|
@rkazants Hi，I wonder why some Ops（such as translate_einsum_op） return vector of Op, like OutputVector translate_einsum_op(const NodeContext& node) {
default_op_checks(node, 1, {"Einsum"});
/*...*/
OutputVector inputs;
for (int input_ind = 0; input_ind < input_size; ++input_ind) {
inputs.push_back(node.get_input(input_ind));
}
auto einsum = make_shared<v7::Einsum>(inputs, equation);
set_node_name(node.get_name(), einsum);
return {einsum};
}but some Ops (such as translate_elu_op) return Op->outputs, like OutputVector translate_elu_op(const NodeContext& node) {
default_op_checks(node, 1, {"Elu", "ELU"});
auto input = node.get_input(0);
auto alpha = node.get_attribute<float>("alpha", 1.0);
auto res = make_shared<v0::Elu>(input, alpha);
set_node_name(node.get_name(), res);
return res->outputs();
} |
|
Hi @awayzjj In the case of On the other hand, some operations can produce multiple output tensors. For example, an operation might produce a result tensor and also a tensor that indicates whether each element of the result is valid. In these cases, the translation function needs to return all of the output tensors, so it returns the In the case of |
|
@siddhant-0707 Thank you for your detailed reply! So operations which only produce a single output tensor like |
|
@awayzjj Yes, that's correct. For PadV2 adding complex type support should be fairly straightforward, you can take a look at Pad complex and replace the |
|
@siddhant-0707 @rkazants I have made several attempts, yet I am still uncertain about how to convert Additionally, I wonder how to print the actual tensor value of the I would greatly appreciate any guidance or suggestions you can offer. Thank you! |
|
@rkazants Hi, would you please provide some advice on the questions I raised above? Any suggestions！ Thank you. |
|
Hi @awayzjj, really sorry for the delay. Good attempts, PadV2 for complex type is a bit tricky case. It is because we should not forget that So I propose complex type is to handle separately without calling Best regards, |
|
@rkazants Hi, thank you for your response! Could you please provide me with some suggestions on how to retrieve the real or imaginary parts of the As its element type is Therefore, I cannot follow the handling method used for |
Hi @awayzjj,
So perform padding for real and imaginary parts separately. Best regards, |
|
@rkazants Thank you a lot for your comments!. I've submit a PR and I'll appreciate it if you'd like to review it. :) |
Closes #22955 ### Details: - separate input into real and imaginary parts and handle them separately by padding real or imag parts of padding value to them and concatenate them finally. --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
…t#22955) (openvinotoolkit#23388) Closes openvinotoolkit#22955 ### Details: - separate input into real and imaginary parts and handle them separately by padding real or imag parts of padding value to them and concatenate them finally. --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
…t#22955) (openvinotoolkit#23388) Closes openvinotoolkit#22955 ### Details: - separate input into real and imaginary parts and handle them separately by padding real or imag parts of padding value to them and concatenate them finally. --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Context
OpenVINO component responsible for support of TensorFlow models is called as TensorFlow Frontend (TF FE). TF FE converts a model represented in TensorFlow opset to a model in OpenVINO opset.
Some audio models use tensors of complex type. Complex type tensor is a tensor that has elements of complex type. For example, 1D tensor with three elements
x = [1+2j, 2, -2j].For supporting PadV2 operation on complex type tensor, you need to extend the corresponding loader for PadV2.
What needs to be done?
The existing loader for PadV2 needs to be extended by propagating
ComplexTypeMarkfrom input to output and to represent output complex type tensor as a floating-point type tensor with auxiliary dimension that concatenates real and imaginary parts of complex tensor.To validate the extension, the corresponding layer test needs to be updated with complex tensor cases.
Here is an example of how to extend
Reshapeloader to support complex type tensors:Since OpenVINO does not have native support of complex tensors, we handle complex type in intermediate layers by representing them as a floating-point type with additional dimension (specially created) to store real and imaginary parts of the original complex tensor so slicing by the last dimension will give either real or imaginary parts:
x[...,0]- real andx[...,1]- imaginary parts.On the first step, we update
default_op_checkswithtrueflag to indicate that loader forReshapeoperation now handles complex tensors:Secondly, we check if complex type mark exists by anticipated inputs. This mark indicates that input tensor of complex type:
Thirdly, we retrieve a floating-point tensor (with additional dimension to store real and imaginary parts) simulating complex tensor:
After that, we implement conversion for
Reshapefor this particular case. Since a floating-point tensor simulating complex tensor has additional dimension equal to 2,we update input target shape by appending
2value and perform reshape on a floating-point tensor simulating complex tensor.Finally, since
Reshapeshould produce complex tensor by output we insert a new markComplexTypeMarkinto the output.To validate support of complex tensors for
Reshape, the new layer test TestComplexReshape was added.Example how to run the layer test:
Example Pull Requests
Shape,Mul,Reshapein [TF FE] Complex type support extended for Separate Bass model. #21477Rolloperation in [TF FE] Support Complex Tensors #20860Resources
Contact points
Ticket
No response
The text was updated successfully, but these errors were encountered: