Cuda Error with LSTM model, after lot of tries. All files provided !

[arnaud-nt2i]

@AlexeyAB
I'm trying to train one of the YoloV3 LSTM models on #3114
But with all of them I got the same error:

CUDA status = cudaDeviceSynchronize() Error: file: C:/Users/nt2i_arnaud_pauwelyn/Documents/darknet-master(02-10)/darknet-master/src/blas_kernels.cu : axpy_ongpu_offset() : line: 741 : build time: Oct  2 2020 - 10:35:39

CUDA Error: an illegal memory access was encountered

My config is that one with the last repo ( but I also tried with August 15 repo and with cuda 10.0)
I add that every model without LSTM works great (train and infer), yoloV3 or YoloV4, sam, pan... with the same dataset.
yolo_v3_tiny_lstm.cfg.txt
train.txt
images samples : sample.zip
obj.names.txt
obj.data.txt

 .\darknet.exe detector train data/obj.data cfg/yolo_v3_tiny_lstm.cfg yolov3-tiny.conv.14 -map -dont_show -cuda_debug_sync -benchmark_layers
 CUDA-version: 10020 (10020), cuDNN: 7.6.5, CUDNN_HALF=1, GPU count: 1
 CUDNN_HALF=1
 OpenCV version: 3.4.0
 Prepare additional network for mAP calculation...
 0 : compute_capability = 750, cudnn_half = 1, GPU: GeForce GTX 1660 Ti
net.optimized_memory = 0
mini_batch = 1, batch = 1, time_steps = 1, train = 0

I am not the only one encountering issue at training lstm : #6531, #6708
What should we do to solve this?

[arnaud-nt2i]

Ok I have found a walkaround, by using "Yolo v3 optimal" here: https://github.com/AlexeyAB/darknet/releases/tag/darknet_yolo_v3_optimal

[aotiansysu]

Setting bottleneck=1 for every conv_lstm layer in yolo_v3_tiny_lstm.cfg works for me:

[conv_lstm]
batch_normalize=1
size=3
pad=1
output=128
peephole=0
bottleneck=1
activation=leaky