Questions about temporal occlusion for labelling and tracking

[haibao-yu]

Hi, thanks for your great dataset!

I notice there are some labelled objects that are totally occluded and there are no lidar points in them. The visualization could be seen the second picture below. And you also note that "A small number of 3d bounding boxes is annotated despite the object being temporarily occluded. For this reason we make sure to filter objects without lidar or radar points in the nuScenes benchmarks. See #366.".

There are two questions:

1. Since the objects are totally occluded, how do you label the occluded 3D objects?
2. Do you regard the occluded trajectory as one trajectory or two trajectories with different tracking ID in tracking task?

Many thanks!

[whyekit-motional]

@yuhb94 I hope these will clear up your queries:

• Since the objects are totally occluded, how do you label the occluded 3D objects?

The annotation tooling used in nuScenes can show all cameras, lidars and radars for the 20-secs scene. Objects are annotated with interpolation. That means that if an object can be seen at 1 secs and at 3 secs, the frames in between are linearly interpolated, even if the object is occluded (note that for evaluation purposes, boxes without lidar/radar points are removed, as there is no chance for a detector to detect these). Of course there can be all kinds of exceptions, e.g. often you see an object through the window of a car, but may not get lidar returns from it.

• Do you regard the occluded trajectory as one trajectory or two trajectories with different tracking ID in tracking task?

Each instance is treated as a single track / trajectory:

nuscenes-devkit/python-sdk/nuscenes/eval/common/loaders.py

Line 146 in 28765b8

tracking_id = sample_annotation['instance_token']