A FLOWR by any other name?

[DavidPratten]

Hi Julian,

This question is inspired by this comment on your recent youtube video.

The semantics and type systems of XQuery/JSONiQ are already very well thought through by some very smart FP guys (e.g. Philip Walder) and there exists a very rich standard library. Better to improve upon that work (I've always felt that XQuery/JSONiQ needed to be re-visited as they are need now more than when they first came out. I've also felt that they were poorly named thus not giving people an idea of how powerful they are and giving people the wrong idea that they were an "XML-thingy".)

How do the goals of Morel align with / or diverge from the goals of XQuery/ JSONiQ?

David

[julianhyde]

I know XQuery reasonably well - having studied it when it was launched, and written one or two programs - but it was not on my mind when I was designing Morel.

I can see why people see similarities. The FLWOR query structure, the ability to do query optimization and therefore query large 'virtual documents' (i.e. databases), the ability to define functions.

Regarding FLWOR. Putting 'from' first is a no-brainer; every query language does that these days, and the SQL folks would love to change it if they could. Morel does not require clauses to be in the rigid order that implied by FLWOR. Erik Meijer encouraged me to make it compositional, so that 'you can always add a new clause at the end of a query.'

I was more strongly influenced by LINQ than XQuery. Before that, I was influenced by PL/SQL: I liked how you can iterate over a query in a cursor FOR loop, was sad that you could not write 'IF EXISTS query', or easily join an in-memory array to a table. After seeing PL/SQL, and some years before LINQ, I created my own language, Saffron, that integrates SQL into Java. I thought it was clever how LINQ unified relational with functional programming and polymorphism, but I thought that LINQ placed too much emphasis on the "builder" paradigm, and drifted too far into the mindset of compilers and too far away from the mindset of query optimization via algebraic rewrites.

In LINQ you can always see where the query ends, because it is enclosed in braces. In Morel, I wanted relations and relational operators to be part of the language the way that integers, floats and arithmetic operators are part of most modern languages. I guess XQuery and JSONiQ achieve that goal - they bring their respective document models, and the operations that build documents, into the core language.

That said, XQuery always seemed too focused on XML to make it as a general-purpose programming language or query language. This is mostly perception: when you see an XQuery program that is generating XML, the program looks a complete mess, and the structure of the program (including line endings) is lost in the XML syntax. And it has to accommodate baroque XML features such as namespaces and text blocks.

I recall that I found XQuery patterns very beautiful. I need to refresh my memory of them. Maybe Morel's case expression (inherited from Standard ML) yields similar elegance. I allow patterns in Morel wherever I can (e.g. from (a, (b, c)) in source) so perhaps they parallel XQuery patterns.

I don't know JSONiQ as well as XQuery. It isn't carrying the burden of XML's data model, and in fact JSON's data model - atoms, records, lists - has become almost the standard data model today. But Morel's data model adds algebraic data types (sum types), functions, and tuples (albeit syntactic sugar for records) and I claim that those are all useful extensions.

What's in Morel that's not in XQuery/JSONiQ? Higher-order functions, functions as values, type variables (i.e. parametric polymorphism), strong typing, and Milner-Hindley type inference.

Higher-order functions are important in Morel. Of course the relational operators (e.g. project and filter) are higher-order functions, but I want users to be able to use higher-order functions on relations (what SQL would call table functions) and to pass functions as values (so that the built-in relational operators are now operating on collections of functions). Higher-order functions allow us to think about computation differently: recursion, for instance, becomes the search for an efficient way to compute the fixed point of a function.

In Morel you can write a program that returns an atomic value (e.g. an int) or a record. In SQL, of course, you can only return a set of records. I would be curious whether an XQuery program has to return a document. In Morel I have been investigating Monoid queries (#69), so that you can apply relational algebra but never materialize the set of rows.

Morel could be extended to support XML and JSON. But probably via library functions that convert XML and JSON syntax to and from Morel's core data structures - lists, records, atoms. Bringing XML and JSON syntax into the Morel language makes programs more difficult to read for someone doing general-purpose computation. (We have made considerable sacrifices in Morel so that programmers rarely need to use types: this makes it look more like SQL or a weakly typed language than Haskell.)

Thanks for asking the question. I am glad that people see echoes of languages that they love in Morel. I hope you can use Morel for some of the same things as those other languages. And that as we discover Morel's strengths and weaknesses, we continue the conversation about where to go next.

[julianhyde]

Jeez. Youtube won't let me add a comment to my own talk. If someone is able to add a comment against https://www.youtube.com/watch?v=smUm3C5q2IM&t=2388s that points to this thread, I'd be grateful.

[GavinRay97]

Jeez. Youtube won't let me add a comment to my own talk. If someone is able to add a comment against youtube.com/watch?v=smUm3C5q2IM&t=2388s that points to this thread, I'd be grateful.

Done 🙂
https://www.youtube.com/watch?v=smUm3C5q2IM&lc=UgwIUZ270SEs-uJyoSN4AaABAg.9U2Gw8DN7YW9V0fVxa2QqK