OnnxConcatFromSequence input_sequence

[busFred]

I have a list of custom model _BinaryRVC. Each _BinaryRVC outputs a n-dimensional row vector. The Reshape in the first command is to convert each n-dimensional row vector into a n-dimensional column vector.
The goal is to concatenate all the n-dimensional column vectors into a n by c matrix, where c is the number of _BinaryRVC in the list.

y_list: List[OnnxOperator] = [
    OnnxReshape(OnnxSubEstimator(bsvc, input, op_version=op_version),
                np.array([-1, 1], dtype=np.int64),
                op_version=op_version) for bsvc in rvc.binary_rvc_list_
]
y_matrix: OnnxOperator = OnnxConcatFromSequence(y_list,
                                                axis=1,
                                                new_axis=1,
                                                op_version=op_version)

However the above code generate the following error:

TypeError: Unexpected type <class 'list'> for input 0 of operator 'OnnxConcatFromSequence'. It must be an instance of Variable (or a string), OnnxOperator, OnnxOperatorItem, numpy.ndarray, coo_matrix).
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Documents/research/sklearn_plugins/test/sklearn_plugins/rvm/test_rvc_export.py in 
      34 """
      35 onx: ModelProto
----> 36 onx = to_onnx(rvc, X_train[:1, :].astype(np.float64), target_opset=13)

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/convert.py in to_onnx(model, X, name, initial_types, target_opset, options, white_op, black_op, final_types, dtype, verbose)
    214         name = "ONNX(%s)" % model.__class__.__name__
    215     initial_types = guess_initial_types(X, initial_types)
--> 216     return convert_sklearn(model, initial_types=initial_types,
    217                            target_opset=target_opset,
    218                            name=name, options=options,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/convert.py in convert_sklearn(model, name, initial_types, doc_string, target_opset, custom_conversion_functions, custom_shape_calculators, custom_parsers, options, intermediate, white_op, black_op, final_types, dtype, verbose)
    160     if verbose >= 1:
    161         print("[convert_sklearn] convert_topology")
--> 162     onnx_model = convert_topology(topology, name, doc_string, target_opset,
    163                                   options=options,
    164                                   remove_identity=not intermediate,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/common/_topology.py in convert_topology(topology, model_name, doc_string, target_opset, channel_first_inputs, options, remove_identity, verbose)
   1200                               type(getattr(operator, 'raw_model', None))))
   1201         container.validate_options(operator)
-> 1202         conv(scope, operator, container)
   1203 
   1204     container.ensure_topological_order()

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/common/_registration.py in __call__(self, *args)
     24             if args[1].raw_operator is not None:
     25                 args[2]._get_allowed_options(args[1].raw_operator)
---> 26         return self._fct(*args)
     27 
     28     def get_allowed_options(self):

~/Documents/research/sklearn_plugins/src/sklearn_plugins/rvm/_onnx_transfrom.py in rvc_converter(scope, operator, container)
    114                     op_version=op_version) for bsvc in rvc.binary_rvc_list_
    115     ]
--> 116     y_matrix: OnnxOperator = OnnxConcatFromSequence(y_list,
    117                                                     axis=1,
    118                                                     new_axis=1,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/algebra/onnx_ops.py in __init__(self, *args, **kwargs)
     99                     Variable, OnnxOperator, np.ndarray, str,
    100                     OnnxOperatorItem, coo_matrix)):
--> 101                 raise TypeError(
    102                     "Unexpected type %r for input %r of operator %r. "
    103                     "It must be an instance of Variable (or a string), "

TypeError: Unexpected type <class 'list'> for input 0 of operator 'OnnxConcatFromSequence'. It must be an instance of Variable (or a string), OnnxOperator, OnnxOperatorItem, numpy.ndarray, coo_matrix).

I also try the following code. Instead of piping entire list into OnnxConcatFromSequence, I unpack the y_list.

y_list: List[OnnxOperator] = [
    OnnxReshape(OnnxSubEstimator(bsvc, input, op_version=op_version),
                np.array([-1, 1], dtype=np.int64),
                op_version=op_version) for bsvc in rvc.binary_rvc_list_
]
y_matrix: OnnxOperator = OnnxConcatFromSequence(*y_list,
                                                axis=1,
                                                new_axis=1,
                                                op_version=op_version)

But this appraoch result in the following error:

RuntimeError: Operator 'ConcatFromSequence' expects a number of inputs in [1, 1] not 3 (expected opset=13, class opset=11)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~/Documents/research/sklearn_plugins/test/sklearn_plugins/rvm/test_rvc_export.py in 
      34 """
      35 onx: ModelProto
----> 36 onx = to_onnx(rvc, X_train[:1, :].astype(np.float64), target_opset=13)

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/convert.py in to_onnx(model, X, name, initial_types, target_opset, options, white_op, black_op, final_types, dtype, verbose)
    214         name = "ONNX(%s)" % model.__class__.__name__
    215     initial_types = guess_initial_types(X, initial_types)
--> 216     return convert_sklearn(model, initial_types=initial_types,
    217                            target_opset=target_opset,
    218                            name=name, options=options,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/convert.py in convert_sklearn(model, name, initial_types, doc_string, target_opset, custom_conversion_functions, custom_shape_calculators, custom_parsers, options, intermediate, white_op, black_op, final_types, dtype, verbose)
    160     if verbose >= 1:
    161         print("[convert_sklearn] convert_topology")
--> 162     onnx_model = convert_topology(topology, name, doc_string, target_opset,
    163                                   options=options,
    164                                   remove_identity=not intermediate,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/common/_topology.py in convert_topology(topology, model_name, doc_string, target_opset, channel_first_inputs, options, remove_identity, verbose)
   1200                               type(getattr(operator, 'raw_model', None))))
   1201         container.validate_options(operator)
-> 1202         conv(scope, operator, container)
   1203 
   1204     container.ensure_topological_order()

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/common/_registration.py in __call__(self, *args)
     24             if args[1].raw_operator is not None:
     25                 args[2]._get_allowed_options(args[1].raw_operator)
---> 26         return self._fct(*args)
     27 
     28     def get_allowed_options(self):

~/Documents/research/sklearn_plugins/src/sklearn_plugins/rvm/_onnx_transfrom.py in rvc_converter(scope, operator, container)
    114                     op_version=op_version) for bsvc in rvc.binary_rvc_list_
    115     ]
--> 116     y_matrix: OnnxOperator = OnnxConcatFromSequence(*y_list,
    117                                                     axis=1,
    118                                                     new_axis=1,

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/algebra/onnx_ops.py in __init__(self, *args, **kwargs)
    105                     "coo_matrix)." % (
    106                         type(a), i, class_name))
--> 107         OnnxOperator.__init__(self, *args, **kwargs)
    108 
    109     newclass = type(class_name, (OnnxOperator,),

~/anaconda3/envs/sklearn_plugins/lib/python3.8/site-packages/skl2onnx/algebra/onnx_operator.py in __init__(self, op_version, output_names, domain, *inputs, **kwargs)
    318             if (len(self.inputs) < self.input_range[0] or
    319                     len(self.inputs) > self.input_range[1]):
--> 320                 raise RuntimeError(
    321                     "Operator '{}' expects a number of inputs "
    322                     "in [{}, {}] not {} (expected opset={}, "

RuntimeError: Operator 'ConcatFromSequence' expects a number of inputs in [1, 1] not 3 (expected opset=13, class opset=11)

I believe the issue is similar to the one I encountered in #703. I think about using onnx.helper.make_sequence function in the onnx library. However, I soon realize both OnnxSubEstimator and OnnxReshape are subclass of OnnxOperator, which is a class defined in skl2onnx library and not a native onnx tensor type. Therefore, I don't think directly pipe y_list to onnx.helper.make_sequence would work.

For this operator (following the example in ONNX specification for ConstantOfShape), you can use this code:

tensor_value = onnx.helper.make_tensor("value", onnx.TensorProto.FLOAT, [1], [5])
cst = OnnxConstantOfShape(shape, value=tensor_value, op_version=opset)

I'll create a PR to make it easier with numpy.

Originally posted by @xadupre in #703 (comment)

[xadupre]

You should use OnnxConcat instead of OnnxConcatFromSequence. You can find an example here: https://github.com/onnx/sklearn-onnx/blob/master/skl2onnx/operator_converters/gaussian_mixture.py#L63. OnnxConcat is ok when the list of inputs has a fixed length. OnnxConcatFromSequence is more for sequence with a variable length but you would need to build a sequence (see https://github.com/onnx/onnx/blob/master/docs/Operators.md#SequenceInsert).

[busFred]

I figured OnnxConcat is the one I should use. But I encountered error described in #711, which @xadupre found is a bug that requires a bug fix.