Hand tracking gestures

[barnabwhy]

Support for pinch and curl gestures as well as emulating the joystick. Somewhat based on #1715 originally but changed down the line.
Some of the vector math for calculating the joysticks values and positions is a bit iffy at the moment, I'm not really sure how to improve it and could do with some help there i reckon. I fixed it :3

To-do list:

• Gestures
  • Binary pinch gestures for buttons (Y/B: Pinch thumb and middle, X/A: Pinch thumb and ring, Menu: Pinch thumb and little on left hand)
  • Scalar pinch gesture for main trigger (Pinch thumb and index)
  • Scalar curl gesture for grip trigger (Curl middle, ring and little)
  • Joystick click (Curl thumb to palm)
  • Time-based hysteresis
• Joystick
  • Activation (Curl all 4 fingers and touch top of hand with thumb)
  • Vertical axis
  • Horizontal axis
  • Deadzone
  • Center in an ergonomic position (Somewhat accounted for by offset settings)
  • Disable trigger pinch gesture while using joystick to prevent interference
• Settings
  • Enable/Disable hand gestures
  • Activation distance for pinch
  • Activation distance for curl
  • Activation distance for joystick
  • Joystick position offset
  • Enable/Disable hand skeleton (Disables finger tracking in SteamVR, doesn't affect gestures)

[barnabwhy]

I'd like to mention I'm happy for others to commit changes if they want, and for any criticism/feedback on the currently implemented gestures or suggestions for better alternatives.

[ccloli]

From my experience, a few things can be fine tuned (and probably needs more setting slots).

The most frustrating thing is the trigger distance is not that reliable. As #256 (comment) mentioned, the accuracy of Oculus hand pinching only works well on thumb + index, and thumb + pinky is the worst one (and may even be treated as thumb + ring). So you probably need to set trigger distance by each finger, an example from myself can be used as a reference, but since everyone's experience may vary, you can try to find better values.

Besides, there're still some tough works to do, though they aren't must-have, just some suggestions from myself.

• Set a trigger time threshold, in case a button triggered many times in a short time, because of the non-accurate recognition. Like for example, a button should be treat as pressed or not, only if it's pinched or released at least 300ms.
• Use gestures only when hand is facing to you, like what Oculus does on left hand menu pinching / right hand oculus button pinching.

Anyway, thanks for your hard working, as it shows a working POC. Hope to see it in release!

[zmerp]

Set a trigger time threshold, in case a button triggered many times in a short time, because of the non-accurate recognition.

I'd prefer a hysteresis-based threshold. I already told @Meister1593 about this and his PR was blocked on him implementing this.

[barnabwhy]

Set a trigger time threshold, in case a button triggered many times in a short time, because of the non-accurate recognition.

I'd prefer a hysteresis-based threshold. I already told @Meister1593 about this and his PR was blocked on him implementing this.

by this do you mean having a closer activation range than deactivation range, or something else?

[Meister1593]

Set a trigger time threshold, in case a button triggered many times in a short time, because of the non-accurate recognition.

I'd prefer a hysteresis-based threshold. I already told @Meister1593 about this and his PR was blocked on him implementing this.

by this do you mean having a closer activation range than deactivation range, or something else?

It's having activation range and then deactivation range after it was activated, instead of just being triggered.
Is it implemented like this at the moment?... i haven't looked into code much.

[zmerp]

@barnabwhy Can you rebase this PR on master? It seems you pulled in one of my commits and the change list got dirty.

[Meister1593]

is there a way to do a/b x/y clicks while holding something? i couldn't make it it but would be nice to have if possible

[barnabwhy]

is there a way to do a/b x/y clicks while holding something? i couldn't make it it but would be nice to have if possible

Tbh I'm not entirely sure how to make it work. In theory you can just touch your thumb to the tip of the relevant finger while gripping but it's definitely a hard tradeoff between a natural grip gesture and ease of using multiple simultaneous inputs

[Meister1593]

Some feedback in general on p4 (pro, 5.7.2 global)
Most crumblesome gestures were joystick (it's really hard to move forward/back with it), and i accidentally gripped quite often out of view. Maybe joystick click also was pretty hard to do for some reason.

[barnabwhy]

Some feedback in general on p4 (pro, 5.7.2 global) Most crumblesome gestures were joystick (it's really hard to move forward/back with it), and i accidentally gripped quite often out of view. Maybe joystick click also was pretty hard to do for some reason.

Currently in the process of making the joystick range configurable. Also The joystick click should be done by curling the thumb to the palm if that helps activate it more reliably?

It is however quite true it's hard to control the joystick accurately, maybe I should increase the default deadzone?

[zmerp]

Looks very good! One last small thing

[zmerp]

Ok that's good enough. Thanks!

[tjuqxb]

Hi, thank you for the work.
I suggest adding the settings to disable/enable joysticks and buttons simulations separately. Joystick gesture is hard to control without physical device in my opinion.