AlaSQL v3

[mathiasrw]

The codebase is a single file monolit at the moment. This makes it hard to use only the parts you need. The structure also makes it hard to introduce new maintainers as things are not aligned with how JS development is done today.

Lets discuss the next version of AlaSQL in this card. Some old ideas has been documented in https://github.com/agershun/alasql/milestone/7 and @noid2 had a great input here: #157 (comment)

I suggest the goals for the next version to be:

• 100% test coverage of the code is a priority
• Speed is a priority
• INSERT, SELECT, WHERE from in-memory is the foundation (no join, sub query) together with user defined functions
• All other features are thought of as add-ons
• Any function will return a promise
• Code is structured as ES module and based on typescript

[noid2]

@mathiasrw Thank you for the interest in my approach.

First I want to clarify that my approach (initial phase) it is focused more on the long term future and vision but still does not neglect topics about performance, quality, library size etc.

Here is a simple presentation:

Core

• JSON Data Tables - Simple mongo like JSON eg. [{id:”FF”, k1:v1, k2:v2 },{id:”FF”, k1:v1, k2:v2 }]

• SQL Query* - support only data query and transformation (Select, Where, Having ,Case, Join, Group By, Order By, Limit ...)

• JS - Data Table Manipulation. JS is very good at performing operations like insert, update, delete, basic query filter.

• Virtual Table/Schema - By default the data structure should be schemaless. But SQL needs a schema and I will code this feature called “virtual schema”. The idea is to have a schema if you need it (manually or automatically). This feature will also cover SQL index, relations, constraints and data validation.

Input Adapters - Integrate data to the standard of JSON Data Tables. Live/Passthrough option available.
Output Adapters - Output query results based on the standard of the Adapter.
Modules - Add features to the core.

*I strongly recommend limiting SQL syntax only to data query for 3 main reasons: 1. Less parsing needed so smaller and simpler parsing library. 2. The real power of SQL is in the querying capability (no doubt about that). 3. JS is much better for the rest of DML features in SQL language. Doing this only with JS will be much more performant also. Another big reason is convenience. Usually users already have or can easily import data to an object, but it's not so easy to transform an object to a SQL insert statement. With JS you can just run db.insert(data_object). Source: I have been writing complex SQL/NoSQL queries for enterprises for the last 10 years to feed my family.

[noid2]

I still have trouble understanding how the current parser works. Do you have any additional documentation for this?
Or any other source I could read to understand better.

[mathiasrw]

@noid2 Thank you for taking the time document how you are thinking. I am sorry that I did not have time this week to go fully into the dialogue. The box you call the core will probably need two more boxes. One for parsing the SQL string -> AST (abstract syntax tree) and one that generates the functions that does the actual work of selecting/filtering from the AST.

There is no additional documentation about the parser. You can have a look at https://zaa.ch/jison/docs to see more about how Jison works, but I am not sure this will help you understand how the parser and the compiling of the function actually works. The model in alasqlparser.jison basically outputs an ATS.

[noid2]

@mathiasrw I have assumed SQL Parsing, Compiling and Executing in the SQL Box.
I have checked other libraries that do SQL parsing but the AlaSQL parser is the most accurate (even if there are some bugs). Thats why I want to understand the current parser.

This could be the reason why AlaSQL is so popular. :)

[mathiasrw]

Lets move the discussion to #1558