Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for glide typing? #3

Open
mahc9kez opened this issue Jan 7, 2020 · 67 comments
Open

Support for glide typing? #3

mahc9kez opened this issue Jan 7, 2020 · 67 comments

Comments

@mahc9kez
Copy link

mahc9kez commented Jan 7, 2020

AOSP keyboard have settings for glide typing but does not included in OpenBoard.

I know it's because glide typing require libjni_latinimegoogle.so the proprietary library (it can be extracted from OpenGApps Optional/swypelibs-lib-arm64.tar.lz).

But maybe consider add this feature to Play Store version by using product flavors? Since play store users probably already have GApps on their phone. and the repo does not need to include the proprietary library, f-droid flavor could just keep do not include glide typing support. Including such feature does not seems break anything if the OS does not have libjni_latinimegoogle.so.

I'm also trying to build LatinIME, but can't get glide typing works for me (but it works on LatinIME in LineageOS). it's a bit strange. so you may make sure at least it works on your devices if are about to support this feature.

@dslul
Copy link
Member

dslul commented Jan 8, 2020

I thought about this, but I'm hesitant to leave this dependency, even on those phones that have Google apps installed. The ideal thing would be to reimplement the swype logic in the library.

@mahc9kez
Copy link
Author

mahc9kez commented Jan 8, 2020

AnySoftKeyboard have swype support which is FOSS, but the last time I tried it, it's far from good as gboard or any of other proprietary swype IME. I think it's not worth the effect to implement this feature for now.

But if you can add swype support and "joystick delete", it would make the keyboard much more usable. Joystick delete is a feature included in gboard but not enabled by default in stable (enabled by default in testing which require GApps to automatically get the config). With joystick delete, it show you a stick when you press the "Delete" icon, you can select words if you immediately swipe left, lift up to delete. or long press to quickly delete.

@machiav3lli
Copy link

Maybe implementing the space swipe gestures(select & delete) could be a good start point. It is already implemented in SimpleKeyboard (https://github.com/rkkr/simple-keyboard).

@mahc9kez
Copy link
Author

oh I didn't know the delete swipe feature in Simple Keyboard before, it's exactly the same thing gboard have, joystick delete is just a visual stick which is not necessary.

@kojid0 kojid0 mentioned this issue Jan 12, 2020
@ralf401
Copy link

ralf401 commented Jan 20, 2020

Moving the cursor by swiping on space would also be very nice.

@dslul
Copy link
Member

dslul commented Jan 20, 2020

Moving the cursor by swiping on space would also be very nice.

This is already implemented together with delete swipe, but the F-Droid build system takes several days to compile the new versions. The Play Store version is already up to date and has those features.

@ralf401
Copy link

ralf401 commented Jan 20, 2020

Moving the cursor by swiping on space would also be very nice.

This is already implemented together with delete swipe, but the F-Droid build system takes several days to compile the new versions. The Play Store version is already up to date and has those features.

Nice, thank you!

@ghost
Copy link

ghost commented Jan 30, 2020

Hi,
Thanks for great work, after trying (unsuccessfully) to extract AOSP Keyboard from Lineage for several hours, it's refreshing to just install it from f-droid.

But about that gliding - is there any workaround I could do by myself to enable glide?
I was trying to:
-Add libjni_latinimegoogle.so to arm64 in-app folder
But it didn't made a difference
-Remove libjni_latinime.so, copy libjni_latinimegoogle.so and rename it to libjni_latinime.so
But it crashed an app
-Add libjni_latinimegoogle.so to system/lib and system/lib64
But it didn't made a difference

I'm not asking you to implement some options including that lib, but maybe you can just don't cut it from AOSP Keyboard?
So when someone paste that lib into app's folder it would work?

@dslul
Copy link
Member

dslul commented Jan 31, 2020

Hi,
Thanks for great work, after trying (unsuccessfully) to extract AOSP Keyboard from Lineage for several hours, it's refreshing to just install it from f-droid.

But about that gliding - is there any workaround I could do by myself to enable glide?
I was trying to:
-Add libjni_latinimegoogle.so to arm64 in-app folder
But it didn't made a difference
-Remove libjni_latinime.so, copy libjni_latinimegoogle.so and rename it to libjni_latinime.so
But it crashed an app
-Add libjni_latinimegoogle.so to system/lib and system/lib64
But it didn't made a difference

I'm not asking you to implement some options including that lib, but maybe you can just don't cut it from AOSP Keyboard?
So when someone paste that lib into app's folder it would work?

I actually disabled swipe from code, so even if you put that library it won't work. If time permits, I want to reimplement the swipe logic (no ETA though).

@ghost
Copy link

ghost commented Feb 11, 2020

@dslul is there a way to re enable swipe? i can re-compile myself.

@dslul
Copy link
Member

dslul commented Feb 12, 2020

@dslul is there a way to re enable swipe? i can re-compile myself.

Just change here the library's name to jni_latinimegoogle and then enable gestures by setting sHaveGestureLib=true

@ghost
Copy link

ghost commented Feb 12, 2020

should i change JNI_LIB_NAME or uncomment the JNI_LIB_NAME2 line?

@ghost
Copy link

ghost commented Feb 13, 2020

changing JNI_LIB_NAME results in a crash.

@dslul
Copy link
Member

dslul commented Feb 13, 2020

changing JNI_LIB_NAME results in a crash.

Do you have that library in your system?

@mahc9kez
Copy link
Author

Set JNI_LIB_NAME to jni_latinimegoogle result in crash here (can't open the keyboard). Normal typing works correctly when uncomment JNI_LIB_NAME2 line, but crash after glide typing each time (the curve was shown but does not input anything when swype).

I'm using LineageOS 17.0 (Android 10) unofficial, maybe it's ROM related, I have tried to put libjni_latinimegoogle.so into system, but the built-in AOSP keyboard always crash when swype.

@dslul or anyone could try to remove libjni_latinimegoogle.so and include the so file in the apk and check if it could work?

app/src/main/jniLibs/arm64-v8a/libjni_latinimegoogle.so

@ghost
Copy link

ghost commented Feb 13, 2020

changing JNI_LIB_NAME results in a crash.

Do you have that library in your system?

of course. i think it said something related to emoji in the logcat; maybe a mismatch in method names? i tried with the library from opengapps for android 10 and 9.

@ghost
Copy link

ghost commented Feb 13, 2020

i've since read the source code of a different AOSP-based keyboard and it seems JNI_LIB_NAME2 is used as a fallback if the normal JNI_LIB_NAME doesn't work, or something like that (A.K.A. if jni_latinimegoogle isn't installed.) OpenBoard doesn't have that logic in the code, and only reads JNI_LIB_NAME

@stoefelz
Copy link

Is there an (un)official solution to get glide typing working?

@ghost
Copy link

ghost commented Feb 28, 2020

Is there an (un)official solution to get glide typing working?

Not in openboard, in other AOSP-based keyboards, yes

@JonnyHaystack
Copy link

Is there an (un)official solution to get glide typing working?

Not in openboard, in other AOSP-based keyboards, yes

Are any of them FOSS?

@ghost
Copy link

ghost commented Feb 29, 2020

Keyboard itself is, but it requires proprietary libraries for swipe to work.

Only OS keyboard with swipe is ASK.

@JonnyHaystack
Copy link

Yeah and ASK sadly has been horrible to use in my experience, even without swipe :(

@JonnyHaystack
Copy link

Does anyone know if Google's proprietary library has any data collection/network accessing code in it?

@maitra
Copy link

maitra commented Sep 22, 2020

I came across the following zip: https://forum.xda-developers.com/android/development/fix-cm-aosp-keyboard-swipe-fix-zip-t3182789#post83552617 but I don't think that it should work anymore, since the binary is for 32-bit and most phones are now arm64, I think. But perhaps it can help as a fix.

lukaszgdula6 pushed a commit to lukaszgdula6/openboard that referenced this issue Dec 21, 2020
@gochev
Copy link

gochev commented Feb 17, 2021

Shame I would love the gliding even though the only option will be to use a proprietary google library, can we have an experiment with this ? In a fork/branch or something just to see how good or bad it is .. maybe with in-house glide typing as well like in ASK.
I believe this is the best keyboard I really miss the glided typing especially for non-english languages where the words are super long like French or Bulgarian.

@Helium314
Copy link
Contributor

Including proprietary closed source libraries in the app is not going to be accepted on F-Droid.

The only way that may work is allowing the user to provide a library via app settings.

@Surendrajat
Copy link

...or maybe just add it to izzysoft's repo.

@Mrnofish
Copy link

Helium314 makes a good point:

Including proprietary closed source libraries in the app is not going to be accepted on F-Droid.

People ask for F-Droid inclusion because their system ensures no difference between published source and apk.

As far as I understand izzysoft 'simply' packages pre-built apks and makes them available as a repo for the F-Droid client.

In short, izzysoft is a convenience service and there would be little difference between that and downloading a manually built apk from github.

Personally I could be OK with installing the library manually, but I suspect this would quickly turn into a chore for those running Lineage, as it'd probably entail reinstalling the library every week, whenever an update package is released.

If the app could download the library from a trusted source and provide the user with a way to trigger the download/install manually, that workaround would be fine with me.

However I wonder if that's actually feasible?

@Mrnofish
Copy link

Mrnofish commented Jun 7, 2022

FTR, I am basically forced to use OpenBoard as it's the least bad, usable FOSS keyboard for languages other than English.

However, it is a very frustrating experience, and the issues list paints a fairly grim picture, of a stagnant application that is missing what might be considered basic QoL features.

This is to say I, and I believe many others who are in the same predicament too, would very much welcome a fork with a more dynamic, pro-active approach that's willing to listen to the wants of those users who employ OpenBoard as a daily driver.

The base is solid, so it's not so much a matter of major overhauls but rather minor tweaks to improve QoL.

@NoahAndrews
Copy link

Why is a fork necessary? I haven't seen any reason to believe that a PR to add glide typing would be rejected.

@Mrnofish
Copy link

Mrnofish commented Jun 7, 2022

Glide typing is not necessarily something that I would consider a basic QoL feature, although it could perhaps be implemented in a relatively simply fashion - but I'm not going to repeat the whole discussions again.

The reason I used this issue is because some folks had already forked OpenBoard, so it seemed like a decent place to send that message in a bottle.

I haven't seen any reason to believe that a PR to add glide typing would be rejected.

Then perhaps you should read this issue from the beginning?

The original developer (still part of the openboard-team) stated their wish was to reimplement glide typing from scratch, doing which in a fashion that is considered good for everyday use, is already a huge task on a good day, and obviously won't ever get done if even much smaller things are left to languish for months on end.

Just to mention one such smaller issue that is nevertheless highly irritant, OpenBoard will learn incorrect spellings and there is no way for the user to fix those mistakes without resetting the whole thing.

Another issue I'm encountering on a constant basis is OpenBoard adding spurious extra blanks when making a certain type of correction.

It does seem that there is a PR for this last issue, which however has been left collecting dust for 3 months and is part of roughly 20 other unresolved PRs, among which some being a year old and maybe more.

All of this, and additionally the slow release cycle, and sparse activity from openboard-team, suggest that there might be not nearly enough dogfooding of OpenBoard.

So, this is about encouraging those folks with a can-do attitude who already forked and actively worked on OpenBoard, to let them know that a better OpenBoard would be greatly appreciated, if they're willing and able to take up the challenge.

@Helium314
Copy link
Contributor

Personally I could be OK with installing the library manually, but I suspect this would quickly turn into a chore for those running Lineage, as it'd probably entail reinstalling the library every week, whenever an update package is released.

Only saw this now, sorry for the late reply:
No, it would not have to be done again unless you completely reset the app. I'm currently using a version that does excatly this (based on https://github.com/wordmage/openboard with additional preference to load a library).

Such a thing may qualify as acceptable for F-Droid, but allowing to load an arbitrary file to be executed is still a security risk...

Why is a fork necessary? I haven't seen any reason to believe that a PR to add glide typing would be rejected.

Well, it would considerable re-arrange the locations of code files (see wordmage fork), and it would require a library to be either distributed with the app (closed source, maybe legal issues) or provided by the user (hacky, security issue), or to actually re-implement glide typing (massive amount of work).

It does seem that there is a PR for this last issue, which however has been left collecting dust for 3 months and is part of roughly 20 other unresolved PRs, among which some being a year old and maybe more.

Most of them are requests for adding dictionaries. I remember those are not merged because they would increase apk size considerably. There also is a PR/branch for allowing to add dictionaries from files that would improve the situation, but it also seems to be collecting dust.

I have the general impression that a) OpenBoard is getting a bunch of new weird bugs/issues with every new Android version and b) the maintainers simply don't have enough time (and maybe new enough Android) to fix those, and even less to add new features.

Actually I have some more changes in my version of OpenBoard, but seeing the slow progress of other PRs I don't want to spend the time necessary to make them ready for a PR (translatable strings, use of preferences instead of hardcoding everything,...)

@odmfl
Copy link

odmfl commented Jul 31, 2022

I notice this commit can help? It seems that the closed library of google is not used.

LMODroid/platform_packages_inputmethods_LatinIME@bd28487

Is it good for something or is it a useless commit?

@wordmage
Copy link

wordmage commented Aug 1, 2022

I notice this commit can help? It seems that the closed library of google is not used.

LMODroid/platform_packages_inputmethods_LatinIME@bd28487

Is it good for something or is it a useless commit?

This can be tested. I'll be giving it a shot after lunch today.

@wordmage
Copy link

wordmage commented Aug 1, 2022

IMG_20220801_121930_714

It works. No Google libraries are added in. I've pushed the necessary changes to my fork, along with the aforementioned commit. This is much cleaner and I personally think it can be shipped with an OSS build of OpenBoard.

@odmfl
Copy link

odmfl commented Aug 1, 2022

IMG_20220801_121930_714

It works. No Google libraries are added in. I've pushed the necessary changes to my fork, along with the aforementioned commit. This is much cleaner and I personally think it can be shipped with an OSS build of OpenBoard.

Fantastic, i was convinced it was a good commit.
I wanted to ask you so is gesture typing enabled by default?
In my opinion, however, an option to activate or deactivate it at the user's choice would be useful.
Make a PR, if not, please also add the option to activate / deactivate it and then finally we will be able to close this issue in the best possible way

@NoahAndrews
Copy link

The new files don't have license blocks, and they include this comment: // Created by msj on 2017/3/9. That same comment and matching code can also be found in this repository, which does not even have a license file: https://github.com/shijieKika/GoogleKeyboardV7. Specifically, this commit (fa1f6d231) is from less than a week after the 2017-03-09 date, so it seems fairly likely that that repository was where the code was originally developed.

Since the oldest source of this code that we know of has no license associated with it, we definitely can't assume that it was developed under an open source license.

Please try to be careful about where you get code from.

@NoahAndrews
Copy link

NoahAndrews commented Aug 1, 2022

@nift4 @iKeramat @shijieKika Can you provide details on where the glide typing code came from?

@nift4
Copy link

nift4 commented Aug 1, 2022

You already found the repo - while there is no license file, I simply assumed it is Apache2 licensed. I'm aware that's technically not how it works, and I don't except anyone to merge that code, I personally am happy with this solution and I don't care if the license is not properly defined. The repo has not been taken down in five years, and the commit history very much looks like the chinese developer did the code himself - and he chose to upload it for the public - that's all i cared about when porting those commits.

@NoahAndrews
Copy link

Also note that the last commit is called change latinime to kikaime, and that the owner of that GitHub account has an @kikatech.com email address. http://www.kikatech.com/ is the website for a closed-source still-maintained Android keyboard. This feels much more like an employee being careless about where they put company code than it does someone developing open source code and forgetting to add a license file.

@nift4
Copy link

nift4 commented Aug 1, 2022

It's probably nothing to merge as is from the POV of an big project like OpenBoard indeed. Maybe someone will reimplement it based on the ideas of this example code, or maybe a fork with the patch will have to do.
I personally don't care, it works, that's all that matters to me.

@nift4
Copy link

nift4 commented Aug 1, 2022

On a slightly related note, most new files are partially modified copies of the typing/ folder variants, not all-new code.

@odmfl
Copy link

odmfl commented Aug 1, 2022

Yes but the published code is open source on github, it has no license and therefore I don't see where the problem lies and in the end it works. This kikatek keyboard is closed source, but the code we are interested in is opensource and there is no license or anything else that forbids to use it

Also note that the last commit is called change latinime to kikaime, and that the owner of that GitHub account has an @kikatech.com email address. http://www.kikatech.com/ is the website for a closed-source still-maintained Android keyboard. This feels much more like an employee being careless about where they put company code than it does someone developing open source code and forgetting to add a license file.

@NoahAndrews
Copy link

NoahAndrews commented Aug 1, 2022

At least by US law (not sure about other jurisdictions) code without an explicit license has all rights reserved, which means you can't use it. Open source is not the default.

@Mrnofish
Copy link

Mrnofish commented Aug 1, 2022

@Helium314 thanks for the answer, very interesting.

I remember those are not merged because they would increase apk size considerably. There also is a PR/branch for allowing to add dictionaries from files that would improve the situation, but it also seems to be collecting dust.

I could name at least another FOSS keyboard that has solved that problem by distributing the languages as optional components.

Sad to see OpenBoard hasn't progressed any, though, it's still my daily driver.

@odmfl
Copy link

odmfl commented Aug 1, 2022

Ok, I didn't know this US law, even if we are not US.
However the solution is one, let's see if the user is still active.
@shijieKika @shijieKika if you are active for pleasure can you give us explanations about it? Or enter a license in your reposity?

At least by US law (not sure about other jurisdictions) code without an explicit license has all rights reserved, which means you can't use it. Open source is not the default.

@wordmage
Copy link

wordmage commented Aug 1, 2022

I did find the licence for the code dubious while working on it. Despite that, the code does allow for some pointers to wiggle the progress on gesture input.

At the end of the day, we can either work with this, reverse-engineer Google's solution (which we now know can be done from GBoard v7), or develop our own. I don't have time to do the last two options, unfortunately — this is the reason why I don't release my fork to begin with: I'm only comfortable with using it for my personal use (and this commit serves a good enough replacement for the aging library for now).

@BBO-repo
Copy link

Hi, for those curious, I built @wordmage's branch and uploaded an apk here: https://github.com/erkserkserks/openboard/releases/tag/v1.4.4-gesture-typing

Works like a charm!
Used on pixel 6A with GrapheneOS, in both French and English languages
Thank so much to all contributors !!!

@rugk
Copy link

rugk commented Sep 13, 2022

@wordmage 's @erkserkserks Could the gesture fork be ported to v1.4.5 of OpenBoard, respectively could the chnages be merged in there?

https://github.com/openboard-team/openboard/releases/tag/v1.4.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests