Add iOS Android pen pressure / tilt support

[HEAVYPOLY]

Describe the project you are working on:
HEAVYPAINT - Multiplatform graphic painting app www.heavypoly.com/heavypaint

Describe the problem or limitation you are having in your project:
pen pressure / tilt not working on Android or iOS
Describe the feature / enhancement and how it helps to overcome the problem or limitation:
add pen pressure/tilt support for iOS/ (iPad pro) and Android (Samsung note 9 pen)
Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
same as how desktop versions work now
If this enhancement will not be used often, can it be worked around with a few lines of script?:
no
Is there a reason why this should be core and not an add-on in the asset library?:
should behave the same as desktop

[ModProg]

I would like to work on this for Android.
What I found so far:

• Pressure is obtainable in MotionEvent.getPressure.
• in Java:
  • This could be stored in arr in GodotInputHandler.java:187 as a 4th value and passed to GodotLib.touch in ln. 207.
  • It should also be passed in ln. 493 & 508
• in cpp:
  • TouchPos needs a pressure field display_server_android.h:43
  • env->GetFloatArrayRegion needs to be called with size 4 java_godot_lib_jni.cpp:246, pressure also needs to be set for tp below
  • in display_server_android.cpp:627 env-> set_pressure() needs to be called. (prob not in ln. 676 as that's for hover)
  • in ln. 570 & ln. 645 ev->set_pressure() needs to be called (not implemented yet)
  • InputEventScreenTouch needs a pressure_field input_event.h:415 and implementation in input_event.cpp:1082.
  • Same for InputEventScreenDrag display_server_android.cpp:613, input_event.cpp:1139 & input_event.h:441.

These would than need to be made availible for GdScript as well I would expect, but I'm not that experienced with the Godot Engine Development yet, so a direction what would need to be done for that would be great.

[HEAVYPOLY]

@ModProg for iOS we are treating pen input as mouse motion events because mouse motion events already have pressure, tilt etc on desktop. I think it would be good to follow this pattern.

[ModProg]

@HEAVYPOLY So you would suggest use mouse for https://developer.android.com/reference/android/view/InputDevice#SOURCE_STYLUS ? I will try that, but we should make sure that that is correctly set on all devices. iOS had force touch for example, I don't know if any android Phones had/have something similar.

Or should pressure on touch be a seperate issue?

[HEAVYPOLY]

Yea I think stylus should be handled as mouse event because that's how it behaves on desktop and iOS, to keep consistency across all platforms. Also because mouse motion already has pressure / tilt properties.

[ModProg]

@HEAVYPOLY should java_godot_lib_jni.cpp get a new method mouse or should touch be used here and the differntiation be done in cpp?

[HEAVYPOLY]

I think so although I'm pretty new to cpp. @m4gr3d might know better.

On iOS we did it like this:
New pencil_press / pencil_drag / pencil_cancelled functions next to touch_press and touch_cancelled in ios godot_view.mm

pencil events sends a regular mouse event with pressure like this:

void OSIPhone::pencil_drag(int p_idx, int p_prev_x, int p_prev_y, int p_x, int p_y, float p_force) {
	Ref<InputEventMouseMotion> ev;
	ev.instance();
	ev->set_pressure(p_force);
	ev->set_position(Vector2(p_x, p_y));
	ev->set_global_position(Vector2(p_x, p_y));
	ev->set_relative(Vector2(p_x - p_prev_x, p_y - p_prev_y));
	ev->set_tilt(p_tilt);
	perform_event(ev);
};

- (void)touchesBegan:(NSSet *)touchesSet withEvent:(UIEvent *)event {
	NSArray *tlist = [event.allTouches allObjects];
	for (unsigned int i = 0; i < [tlist count]; i++) {
		if ([touchesSet containsObject:[tlist objectAtIndex:i]]) {
			UITouch *touch = [tlist objectAtIndex:i];
			int tid = [self getTouchIDForTouch:touch];
			ERR_FAIL_COND(tid == -1);
			CGPoint touchPoint = [touch locationInView:self];

			if (touch.type == UITouchTypeStylus) {
				OSIPhone::get_singleton()->pencil_press(tid, touchPoint.x * self.contentScaleFactor, touchPoint.y * self.contentScaleFactor, true, touch.tapCount > 1);
			} else {
				OSIPhone::get_singleton()->touch_press(tid, touchPoint.x * self.contentScaleFactor, touchPoint.y * self.contentScaleFactor, true, touch.tapCount > 1);
			}
		}
	}
}

[m4gr3d]

cc @thebestnom for feedback.

[pfeodrippe]

One way that I've found to differentiate between the input type is that InputEvent.get_device() returns 0 for the apple pencil, while it returns -1 for the finger. Is this the best way to know it?

[akien-mga]

Android support was implemented in godotengine/godot#80644

iOS would still be missing I suppose. Would be nice to have feature parity in 4.2 if we can (cc @bruvzg).

[bruvzg]

Support for iOS is implemented in godotengine/godot#70482

[Calinou]

Closing, as this is now implemented on both mobile platforms as requested in the proposal.

(Pen pressure/tilt should be functional on desktop platforms in 4.1 already.)