Indexing error in RPNLossComputation.prepare_targets

[miguelvr]

🐛 Bug

File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/rpn.py", line 100, in forward
    return self._forward_train(anchors, objectness, rpn_box_regression, targets)
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/rpn.py", line 119, in _forward_train
    anchors, objectness, rpn_box_regression, targets
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/loss.py", line 91, in __call__
    labels, regression_targets = self.prepare_targets(anchors, targets)
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/loss.py", line 62, in prepare_targets
    labels_per_image[~anchors_per_image.get_field("visibility")] = -1
RuntimeError: The shape of the mask [5118] at index 0 does not match the shape of the indexed tensor [0] at index 0

I am getting this error and I have no idea why... At some point in RPNLossComputation.prepare_targets, matched_targets is a BoxList with num_boxes=0, resulting in an empty tensor in labels_per_image and subsequently in a indexing error down the line.

maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/loss.py

Lines 50 to 62 in ced10f2

	def prepare_targets(self, anchors, targets):
	labels = []
	regression_targets = []
	for anchors_per_image, targets_per_image in zip(anchors, targets):
	matched_targets = self.match_targets_to_anchors(
	anchors_per_image, targets_per_image
	)
	matched_idxs = matched_targets.get_field("matched_idxs")
	labels_per_image = matched_idxs >= 0
	labels_per_image = labels_per_image.to(dtype=torch.float32)
	# discard anchors that go out of the boundaries of the image
	labels_per_image[~anchors_per_image.get_field("visibility")] = -1

Environment

Docker image

PyTorch version: 1.0.0.dev20181119
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.4 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: 
GPU 0: Tesla V100-DGXS-16GB
GPU 1: Tesla V100-DGXS-16GB
GPU 2: Tesla V100-DGXS-16GB
GPU 3: Tesla V100-DGXS-16GB

Nvidia driver version: 384.145
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.2
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a

[fmassa]

This is a duplicate of #31
The error happens because you have an image in your dataset that has no ground-truth targets.

[miguelvr]

@fmassa I've checked and that is not true. All my images have annotations, there must be a transformation that is creating empty BoxLists

[fmassa]

Maybe one of your images have an empty annotation, like [[0, 0, 0, 0]]?
This gets removed in here.
Another possibility is that you are doing random crops in the image, and in one of the crops you don't have any GT.

[miguelvr]

Maybe one of your images have an empty annotation, like [[0, 0, 0, 0]]?
This gets removed in here.

That is not the case.

Another possibility is that you are doing random crops in the image, and in one of the crops you don't have any GT.

This would make a lot more sense, but it should still be verified.

[fmassa]

The error points down to the fact that there is not GT for one image that has been fed, that is kind of clear from the error message and there is BTW and open PR to improve this error message in #37

If you are adding random crops in your transforms, that is probably the culprit.

[miguelvr]

Is there an option to not add random crops? I'm not doing it explicitly

[fmassa]

We don't add random crops by default (this option is not even present in the current codebase actually)