Concat of empty array and array with data crashes Mxnet

[fab-san]

Description

Concatenating an empty array with an array with some data crashes Mxnet. A similar script in Numpy works just fine (np.concatenate([np.empty((0, 4)), np.array([[1,2,3,4]])], axis=0)).

Environment info (Required)

----------Python Info----------
Version : 3.6.8
Compiler : GCC 5.4.0 20160609
Build : ('default', 'Dec 24 2018 19:24:27')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 19.1
Directory : /usr/local/lib/python3.6/dist-packages/pip
----------MXNet Info-----------
Version : 1.4.0
Directory : /usr/local/lib/python3.6/dist-packages/mxnet
Commit Hash : a03d59e
Library : ['/usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so']
Build features:
No runtime build feature info available
----------System Info----------
Platform : Linux-4.15.0-54-generic-x86_64-with-Ubuntu-16.04-xenial
system : Linux
node : 776b204724f1
release : 4.15.0-54-generic
version : #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019
----------Hardware Info----------
machine : x86_64
processor : x86_64
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 158
Model name: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
Stepping: 9
CPU MHz: 4500.197
CPU max MHz: 4500.0000
CPU min MHz: 800.0000
BogoMIPS: 8400.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0013 sec, LOAD: 0.6299 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0015 sec, LOAD: 0.3904 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0016 sec, LOAD: 0.3391 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0012 sec, LOAD: 0.5588 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0317 sec, LOAD: 0.3013 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0013 sec, LOAD: 0.0779 sec.

Package used (Python/R/Scala/Julia):
I am using Python 3.6

Error Message:

  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-31-ebeb526ad88a>", line 1, in <module>
    mx.nd.concat(a,b, dim=0)
  File "<string>", line 68, in concat
  File "/usr/local/lib/python3.6/dist-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/usr/local/lib/python3.6/dist-packages/mxnet/base.py", line 252, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [21:19:35] src/imperative/./imperative_utils.h:124: Check failed: infershape[attrs.op](attrs, &in_shapes, &out_shapes) 
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(+0x230512) [0x7f58dae4b512]
[bt] (1) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(+0x230b28) [0x7f58dae4bb28]
[bt] (2) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(mxnet::imperative::SetShapeType(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, mxnet::DispatchMode*)+0x911) [0x7f58ddbc2ea1]
[bt] (3) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&, std::vector<mxnet::NDArray*, std::allocator<mxnet::NDArray*> > const&)+0x2f1) [0x7f58ddbcca71]
[bt] (4) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(+0x2ebe26b) [0x7f58ddad926b]
[bt] (5) /usr/local/lib/python3.6/dist-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x6f) [0x7f58ddad982f]
[bt] (6) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f58f7efce40]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f58f7efc8ab]
[bt] (8) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2be) [0x7f58f8110b9e]
[bt] (9) /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x10da4) [0x7f58f810fda4]

Steps to reproduce

a = mx.nd.empty((0, 4))
b = mx.nd.array([[1,2,3,4]])
mx.nd.concat(a,b, dim=0)

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Bug

[leleamol]

@mxnet-label-bot add [NDArray]

[leleamol]

I could reproduce this issue. As per the call stack below the operator "Concat" fails to infer the shapes. I also noticed that the operator "_zeroes" fails to infer shape when we try to intialize ndarray as follows:
m = mx.ndarray.zeros((0,4))
This is already captured in the issue #15766

I am curious to know what is the significance of having "0" as one of the shape dimension?

Here is the call stack for "concat" operation

>>> mx.nd.concat(a,b, dim=0, k)
  File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument
>>> mx.nd.concat(a,b, dim=0, out=k)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 68, in concat
  File "/usr/local/lib/python3.7/site-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/usr/local/lib/python3.7/site-packages/mxnet/base.py", line 253, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [17:44:00] src/imperative/./imperative_utils.h:145: Operator Concat inferring shapes failed.
input shapes:
[-1,4]
[1,4]
output shapes:
[1,4]
operator attributes:
num_args : 2
dim : 0

Stack trace:
  [bt] (0) 1   libmxnet.so                         0x0000000112ef8929 mxnet::op::NDArrayOpProp::~NDArrayOpProp() + 4473
  [bt] (1) 2   libmxnet.so                         0x0000000114477b1f mxnet::imperative::SetShapeType(mxnet::Context const&, nnvm::NodeAttrs const&, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> > const&, mxnet::DispatchMode*) + 8767
  [bt] (2) 3   libmxnet.so                         0x000000011447483c mxnet::Imperative::Invoke(mxnet::Context const&, nnvm::NodeAttrs const&, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> > const&, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> > const&) + 716
  [bt] (3) 4   libmxnet.so                         0x00000001143ba48e SetNDInputsOutputs(nnvm::Op const*, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> >*, std::__1::vector<mxnet::NDArray*, std::__1::allocator<mxnet::NDArray*> >*, int, void* const*, int*, int, int, void***) + 1582
  [bt] (4) 5   libmxnet.so                         0x00000001143bb1d0 MXImperativeInvokeEx + 176
  [bt] (5) 6   _ctypes.cpython-37m-darwin.so       0x00000001096b335f ffi_call_unix64 + 79

Based on callstack above, I noticed that the input_shape changed from [0,4] to [-1,4].
When I looked at the code of "Concat" operator, the output "size" for "dim=0" came out to be 0 (as a result of concat of "-1" and "1"). I have limited understanding of this code but this could be the cause of crash.

[haojin2]

Fixed in new mxnet.np module now:

>>> from mxnet import np, npx
>>> npx.set_np()
>>> np.concatenate([np.empty((0, 4)), np.array([[1,2,3,4]])], axis=0)
array([[1., 2., 3., 4.]])

@fab-san

[haojin2]

due to the in-activity, closing the issue due to fixed in mxnet.np