Train an action classification model with dataset containing just the object to track

[JasseurHadded1]

Hello,

If we want to create an action recognition model, let's suppose that we have an object detection model that detect persons.

Instead of using our classic dataset in the action recognition model, I have thought about modifying this dataset by just putting the persons and remove the background by using our object detection model.

To keep the temporal features, my modified dataset will contains frames that are all blacks except the person boxes, and the aim of this is to make the model concentrate better on the person.

Do you think is a good idea?

Thanks,

[mzhadigerov]

I tried this. Results were bad.

[mzhadigerov]

@JasseurHadded1 No, there is no discontinuity. You track an object and once you lose focus, you save those frames as a separate video and continue with the next detection/tracking etc. You will end up with a list of small videos. Based on how your tracking algorithm works, it should be fine. I can send you a demo video, If you give me a link to send

[mzhadigerov]

or I can show you my screen, If you give me your discord or whatever

[JasseurHadded1]

I think we are not talking about the same thing. I talk just about one action within a single video. From what I understand, you're taking the object (let's say it's a person), cropping the person as a rectangle, and then resizing the frame. If my understanding is correct, the frame i+1 will not be continuous with the frame i. (I mean when you play the video, it won't appear fluid and real)

[mzhadigerov]

• @JasseurHadded1 yes, I re-read your original question and we are talking about different things. You asked about removing background and replacing with black colour for training, while I wrote about removing background completely for inference. I apologise for the confusion. And yes, frame transition is not always very smooth, due to sizes of bounding boxes, If that's what you mean by discontinuity. What I meant by not having discontinuity is that there are no black frames between frames. In my case it worked tho - I am able to get proper predictions when a particular action happens. Though, I'm having a lot of false positives. Not sure If that's because the frame transition is not smooth or it is just a problem with the model itself

[JasseurHadded1]

It's clear now. Thank you for the discussion, and good luck!