Mouse absolute coordinate mode (direct touch, or pointer-to-touch-location)

[gilnoh]

What this pull request is about?

This pull request implements an additional mode of passing mouse movement. When enabled, it passes the mouse motion in absolute coordinate: this changes in-dos pointer behavior of mouse pointer directly go to touched point, or iPad's pointing device point.

Essentially, this pull request is doing the same thing to that of pull request #60. But this pull request is more polished, and more up-to-date with current code base. Absolute coordinate mode is a small extension mode of current mouse handling code in SDL_uikitview, and it support all current features (right click, pointing device) in the same manner.

My motivation was originally on supporting better HW mouse. With this patch, pointing devices (ipad magic keyboard, or bluetooh mouse) finally behaves quite close to a native mouse. For example, there would be no more strange pointer-stuck-in-the-middle due to iPad pointer hitting screen limit. However, handling direct touch (pointer to touch location) was also carefully checked and implemented. After some days of testing, I believe this is more or less feature complete. Now I am making it a pull-request for further discussion, and/or improvements.

How to use? (user perspective)

The patch adds a settings group named "mouse absolute coordinate", which has single toggle switch (enables absolute mode), and two scale factor selection button (X scale and Y scale). By switch on "Enable" toggle, you can start using the patched absolute coordinate mode.

Touch will then bring the mouse to the touched location (if scale factor is correct, see below). Button behavior is the same to normal (relative) mode; where you can do left click by tab, or right click by dual touch tab, and so on, depending on your other settings. First touch (primary touch) will always have the mouse pointer. If you hold and drag, the pointer will also follow your finger location.

Note that, each dos program may need different scale factor. If you don't know about a program's mouse scale -- It is a good to start with 1x, since even if that isn't the right factor, you will see the mouse pointer and have some idea what additional factor you need to make it sync with your touch point.

How is it implemented? (code change overview)

• SDL_Mouse actually supports both absolute mode and relative mode. However, dosbox's mouse event handling part (sdlmain::HandleMouseMotion) picks up only the relative values, and do not really sync the reported absolute coordinate of SDL mouse.
• This patch changes dosbox mouse event handling code to respect absolute value, if the passed event was generated by a SDL_Mouse in absolute value mode. The event (SDL_MouseMotionEvent) and event sender (SDL_SendMouseMotion) has been updated to hold relative_mode flag in the event data structure.

With the above data path enabled, then actual absolute coordinate mode logic has been added to SDL_uikitview:

• ensureSDLMouse method is updated to ensure the single SDL_mouse's relative_mode value to relative / absolute according to the setting.
• added a method sendMouseCoordinate, which sends absolute coordinate (vs. movement of sendMouseMotion)
• touchesBegan, touchesMoved, touchesEnded and pointerInteraction has been updated to hold additional logic of absolute modes.

[convolutionbbs]

I’d support this mode if it helps with bluetooth mouse compatibility - especially if it’s a toggle.

[gilnoh]

@convolutionbbs That is the case. The patch does improve bluetooth mouse compatibility, and yes, it is a toggle in the settings. With a correct scale factor, in-dos mouse pointer will sync to that of iOS pointer, which makes the mouse handling more natural and consistent.

---

I've tested the patch in my iPad pro (with touch pad magic keyboard, and then with a bluetooth mouse), on various classic RPG games (Might and Magic series, Eye of the beholder series, and so on). It would be nice to hear how the patch feels by other users on other games.

In my case, the improved mouse handling made playing EOB series in a more fluid fashion. Also, direct touch enabled a different way of enjoying the game like that of Might and Magic 3, where I now find myself playing mostly with keyboard and direct touch; e.g. play usual commands such as attack and move via keyboard, and if a selection comes up on screen, I don't bother the mouse, and just touch the screen to select and proceed, etc.

The one part that feels incomplete is that, you shave to select a scale factor manually. e.g. M&M 3 requires (1x, 1x) scale factor, while EOB requires (2x, 1x) factor, etc. It would be nice if someone can figure out how this can be set automatically -- but I am not sure it is even possible...

[convolutionbbs]

Sounds good - I’d be happy to test if it makes its way in to the app store version or perhaps a test flight version (if that exists). I seem to have trouble building from source, so having someone more experienced then I do that part would be best for me.

[litchie]

Sorry I didn't see this earlier, it looks great to me. I'll do some test.

One thing I am not sure is that the settings perhaps should be saved in dospad.cfg so that packaged games (.idos) can have some presets automatically.

[litchie]

@gilnoh I might have successfully got rid of the scaling.
Can you please try the branch pr110: 939b959

For the games I tried, it does the auto scaling all right.

[litchie]

@convolutionbbs I'll do a test flight build for you

[gilnoh]

@litchie That's really nice! I guess the normalization should work well and sound. I will test the branch on my side and update how it feels on my side, shortly!

[gilnoh]

I have tested the branch on various games that I tested for the update —EOB series, M&M series and some other New World computing RPG games —, it worked perfectly for those games. I think the logic seems to be stable enough to be used, and you can remove the ugly “X/Y factor” drop-down in the settings! :-) I am very glad that you have fixed this final piece and made it complete!!!

Please go ahead with your branch and finish the feature. I am pretty sure many dospad users will like it!

[litchie]

Thank you very much for your hard work. What a gift for 2021!

[convolutionbbs]

Yes thank you! Will this come out in an update to the App Store version?

[litchie]

Yes, of course, after some beta testing.

[convolutionbbs]

Wow, tested this and it works fantastic. Another great addition to this software on the platform. I also tested using the trackpad on the magic keyboard and the cursor followed the pointer 1:1. I have double tab for right click enabled, so I can double tab (not double click) the trackpad for right click and this works great for adventure games.

Thanks again!

[convolutionbbs]

So most of the Sierra games work well with this feature, but I couldn’t get the mouse to respond at all in Quest For Glory 4 CD. If I toggle it off the mouse responds, but as soon as I toggle it back it is just frozen. I tried both extremes 1x and 8x for both axis and it didn’t take. I know this game is fairly buggy, but thought I’d share my experience in case the root cause also causes other games to not work.