Consider using a Trie for dictionary storage

[sspanak]

Instead of the flat storage in SQL, try using a Trie. It may be much faster and use less space. When switching, ensure the suggestions still have priorities.

Educational info:

• Succint Data Structures
• Compressing Dictionaries

A DAWG may be even a better option, because it allows sharing common intermediate nodes. For example:

t | o | p
  | a |
  | i |

This may be useful for compressing the assets. Here is a nice implementation in Python.

[Clam-]

You may find that SQLite is quite optimized at searching and storing due to the underlying tree nature of its internals.
Historically the very first version of the app I attempted to use a trie-like data structure. I wasn't very successful, since I wasn't sure how to optimally store it in memory vs disk and ended up consuming all memory trying to load a dictionary haha.
SQLite does all that messy work for you, however even in saying all that I would be very interested to see what you end up creating.
Good luck!

[sspanak]

You may find that SQLite is quite optimized at searching and storing due to the underlying tree nature of its internals.

This is not really something absolutely necessary. SQLite works just fine. Even in debug mode it loads the suggestion list in less than 10 ms, so there is no need to replace it.

I was not familiar with this data structure. I found about it accidentally, while I was working on another tree in my real job. The demo in the first link seemed quite impressive, so I though I could experiment with it, when I have enough free time. But of course, I immediatelly realized that a relational database is not an optimal storage for this data structure and I knew I had to find a better one.

Anyway, my interest in this could be classified as purely scientific.

Historically the very first version of the app I attempted to use a trie-like data structure. I wasn't very successful, since I wasn't sure how to optimally store it in memory vs disk and ended up consuming all memory trying to load a dictionary

That's strange. According to many sources this is the data structure for dictionaries. It's the most optimal one and it has been used in early phones that had very little processing power and memory. Who knows... you may have not been successful, because of the way Java works... I may find one day!

[github-actions]

This issue is stale because it has been open for 100 days with no activity.

[sspanak]

This is an interesting academic problem, but a more realistic approach is replacing the database as described in #366. If performance issues still persist, this can be reopened.