WIP: initial browser platform

[brodycj]

Basic functionality working with browser platform (ref: #297). This is basically a special version of SQLitePlugin.js which simulates the sqlite plugin functionality on the Chrome browser. Note that this assumes that the Chrome browser will not remove the deprecated Web SQL implementation anytime soon.

NOTICE that the echoTest, selfTest, and deleteDatabase functions are not implemented.

Some tests were disabled or commented out due to timeouts. A number of tests will need to be adapted to deal with the behavior on the browser platform.

[arcadius]

Hi @brodybits
I applied these changes and things seems to be working well in Chrome.

Please, when is this going to be released?

Thanks for the great work.

Arcadius

[brodycj]

Thanks @arcadius for your interest. The thing is that there are some differences in behavior between this plugin and WebKit Web SQL on chrome listed in https://github.com/litehelpers/Cordova-sqlite-storage#deviations, which could trip developers who develop on Chrome browser and then run on a mobile platform. I will probably need a couple months or so to clear the most serious deviations before releasing. Priority feature support goes to paid customers, please contact sales@litehelpers.net for more details.

[arcadius]

Thanks @brodybits for the detailed info 👍

[saikirangit]

Hi @brodybits any idea when this would be released?

Thank you for such a wonderful plugin.

[willianbs]

How can I use this feature?
Thanks in advance!

[brodycj]

You should be able to use Git to checkout my cb-browser-wip branch, use it as-is, rebase onto default storage-master branch, or merge into storage-master branch. Note that the cb-browser-wip branch is currently missing a fix needed to avoid possible data loss in case of multi-page applications as discussed in BUG #666. I hope I get a chance to rebase the cb-browser-wip branch to include the updated workaround solution to bug #666 in the near future.

P.S. The cb-browser-wip still has a number of TODO items, does not pass all tests, and should be regarded as experimental at this point.

[lasterra]

Hi, in my case I need a better than chrome implementation of SQLite in browser. My application use attach database an cross database joins functionality and that is not possible on chrome sqlite implementation.

So i started to develop the browser platform for the pllugin using https://github.com/kripken/sql.js/.

One of the thing i use it's the cordova-proxy used in cordova-plugin-file plugin to capture all cordova calls to redirect that calls for browser platform. It means that SQLitePLugin.js doesn't need to be modified and the proxy work is the same as android/ios plugin.

@brodybits Do you think it is interesting for the project?

[brodycj]

So i started to develop the browser platform for the pllugin using https://github.com/kripken/sql.js/.

From what I saw sql.js only stores in a memory database, does NOT read or write to actual sqlite3 database files. Please share a clear pointer in case I am mistaken here.

One of the thing i use it's the cordova-proxy used in cordova-plugin-file plugin to capture all cordova calls to redirect that calls for browser platform.

Sounds interesting but I do not see this. It would be great if you can share a pointer.

@brodybits Do you think it is interesting for the project?

Probably not due to the extra overhead and complexity.

Please report here or in private to brodybits@litehelpers.net if you publish a fork or new project using SQL.js. I would be happy to link to it if it would be sufficiently useful for some other users.

[lasterra]

My idea is to execute selenium test on the browser. This is why I started to work in cordova platform browser.

SQL.js is fast enough, work well, and support complex sqlite querys like attach. It consume a lot of memory, in my case 500mb with 200mb database.

It's true that it is an in memory database, but you can export and import to browser persistent filesystem quite fast.

The code to support browser platform is easy and does not affect to other platforms like android or iOS.

I don't know how to share my code, maybe a gist ?

[brodycj]

Hi @lasterra,

It sounds good whether you want to share your implementation as a gist or as a whole project on GitHub. (It may help some more people if you share it as a fork or new project on GitHub.) I will probably want to link to it, in case it may help some others.

Also, the memory usage does not surprise me.

[DanielSWolf]

My colleagues and I just discussed the question of Web SQL vs. sql.js. To get an overview, I assembled a feature matrix:

Feature	Web SQL	sql.js	Comment
Works in any browser	-	✔	Web SQL is only supported in Chrome and Safari. sql.js runs in any modern browser.
Future-proof	-	✔	Web SQL has been deprecated since 2011. Accordingly, browser support will likely be dropped in the future.
Persistent	✔	-	Web SQL databases are persisted by origin and name. sql.js databases are in-memory only. They can be loaded from and persisted to integer arrays, though. So as long as the database file doesn’t grow too big, it might be feasible to persist it using other technologies such as IndexedDB.
Up to date	✔	✔	Chrome’s Web SQL implementation wraps the up-to-date SQLite 3.22.0. sql.js is based on the up-to-date SQLite 3.22.0.
Full SQLite syntax possible	-	✔	While both approaches use SQLite under the hood, Web SQL applies some restrictions. For instance, out of the five syntaxes for passing arguments offered by SQLite, only one (?) is supported.
Extensible	-	✔	sql.js is open source, so any aspect of its JavaScript interface can be modified. It even allows adding SQL functions defined in JavaScript.
Non-blocking	(✔)	(✔)	Web SQL blocks "quite a bit" in Chrome, but not in Safari. sql.js is synchronous by default, but can be used in a web worker.
Accessible from web workers	-	✔	WebSQL support inside of web workers has been dropped in Chrome 39 and in Safari 9. sql.js works in web workers.
Low memory consumption	✔	-	Web SQL holds the database in a file and has minimal memory overhead. sql.js holds the entire database in memory. I loaded several large databases, and consistently found that the memory required matched the size of the database plus an overhead of 3-5%.
Fast	✔	✔	I ran multiple queries against a 62 MB production database containing a table with 156,866 rows. To measure native SQLite performance, I used SQLite Expert Personal; to measure sql.js performance, I used the online demo running in Chrome on the same machine. The tests consistently show that sql.js is only 30% slower than native SQLite.
Maximum database size	5 MB - unlimited	ca. 500 MB	The Web SQL standard doesn’t specify a maximum database size. Safari warns for database sizes over 5 MB, Chrome apparently doesn’t. For sql.js, there are two limiting factors. First, memory. On my machine, I managed to open a 499 MB database file in Chrome; larger files failed to load. Second, the persistence system. IndexedDB allows multiple GB of data, which is much more than can be kept in memory.

For our team, the decision was obvious. sql.js covers everything we need, while Web SQL is only supported on two browsers, and even these two may remove support at any time. We would very much like to see a port to sql.js!

Other teams may have very different requirements, of course. If persistence is a key requirement, Web SQL may look better.

[brodycj]

@DanielSWolf thanks for the feedback, you already got a couple thumbs up. I have the following remarks:

• "Accessible from web workers" - Could be tricky with SQL.js if you want to access the same (in-memory) database from multiple threads, may need to be coordinated from the main thread with special library scripts imported in case of worker access.
• I would consider the memory usage (SQL.js) and maximum database size (both Web SQL and SQL.js) to be major limitations that would need to be tested and documented.
• In-memory (lack of persistence) in case of SQL.js could be a nice advantage for developers who want to start clean whenever testing. (Web SQL does not have in-memory database option.)
• Limited (very limited) SQL.js persistence is possible ref: https://github.com/kripken/sql.js/wiki/Persisting-a-Modified-Database; better SQL.js persistence should be possible by using File API
• cordova-sqlite-ext pre-populated database feature would LIKELY be feasible with SQL.js (I cannot guarantee this without some probably very basic testing)

In case of cordova-sqlite-storage and cordova-sqlite-ext I generally prefer to keep the binary and other bulky dependencies (such as sqlite3.[hc]) in a separate npm package that is already added by a before_plugin_install hook script. It should be no problem for this to work with SQL.js but I would like to test it before making any promises on this plugin version. Using SQL.js should be no problem on cordova-sqlite-evcore-extbuild-free and upcoming evmax plugin versions, though it may be a slightly breaking change.

[brodycj]

I started working on implementation of browser support using SQL.js in #774, discovered something I had overlooked: browser support using SQL.js will NOT work across page changes in a multi-page app due to lack of actual persistence.

Recommendation for those who need persistence on browser platform and those who want browser platform support TODAY is to use brodybits / sql-promise-helper; how to use brodybits / sql-promise-helper on browser platform is described in brodycj/sql-promise-helper#4.