Interactive Face Recognition Demo

[Luxonis-Brandon]

Start with the why:

In many applications (including interactive robotics) it is beneficial for the system to be able to know which person it is interacting with (even if it doesn't know this person by personally-identifiable information like a name). And the flow of face detection, to then facial landmark detection, to then re-identification is a great demonstrator/reference app for other such similar applications (including recognizing specific/unique vehicles, for example) and more broadly how to build such flows with the Gen2 Pipeline Builder #136. So it serves as a good reference from which other similar flows can be built (such as, this car drove by X number of times, or that car keeps parking illegally).

Move to the how:

Intel provides an example of this flow using OpenVINO on a computer here. We initially built a hard-coded version of this (video here in architecting the Gen2 Pipeline Builder #136, to guide what nodes/systems would be needed.

Now that the Gen2 Pipeline Builder is nearing formal release, it is likely already possible to implement the whole flow using this Gen2 API, so that it can be modularly changed/added-to/reworked for other applications.

Move to the what:

Implement the Interactive Face Recognition Demo in the Gen2 Pipeline Builder architecture.

[dhruvmsheth]

Might be a bit late to add to this. Since the Gen2 API is yet under active development, the model facial-landmarks-35-adas-0002 can be integrated, to provide a base framework for many Advanced Driver Assistance Systems like the once you mentined above, and one of the others mentioned in pyimagesearch, an application of Eye Aspect Ratio. can be built on top of 35 Facial landmarks OpenVINO Model.

There are a few scripts built on top of the existing 35 facial landmark model to implement that application, using Myriad X, there NCS2 but could be changed to OAK or OAK-D here.

Repository - https://github.com/incluit/OpenVino-Driver-Behaviour

Most of the implemented applications are ADAS applications, implementing EAR, and YAWN detection using facial landmarks. The landmarks-regression-retail-0009 is already implemented in the Traingulation Visualisation example, so that could be used as a framework for only basic facial landmark detections.

Here's a graph displaying recorded FPS on the repo given above, reaching 22 FPS on NCS2, and could go higher to 30 on OAK-D. On the existing Gen1 Pipeline, it reaches 30FPS.

These are secondary Models deployed on face-detection-adas and retail models

Implemented Examples -

[Luxonis-Brandon]

Thanks yes we should definitely add this. Thanks for bringing it up! And if you feel like taking a stab, please do!

[Luxonis-Brandon]

Done in https://github.com/luxonis/depthai-experiments/tree/master/gen2-face-recognition and a bonus https://github.com/luxonis/depthai-experiments/tree/master/gen2-fatigue-detection. Thanks to ArduCam for both!