2009 09 02 how to write a linq provider the simple way

Published on September 2nd, 2009 at 8:31

How to write a LINQ provider - the simple way, Part I

LINQ – Language-Integrated Query – is an interesting technology. It’s a declarative, concise, and quite natural way of specifying queries for different query sources, and it even comes with syntactic sugar for the two big .NET programming languages C# and VB.NET.

What’s especially interesting about it, though, is how extremely difficult it is to write a sensible provider for additional query sources. Oh, of course, it’s easy to write a simple one, one which only supports the Select and Where query methods. And maybe a little OrderBy every now and then.

But it gets incredibly hard once you try to add all those little things that make up the whole of LINQ: multiple from clauses (SelectMany), joins, group joins, groupings and aggregates, subqueries, and so on. And when, some day, you’ve actually understood all of those constructs with all of their subtleties and consequences, then there comes the realization that to actually execute a LINQ query against a queryable system (such as a SQL database, for example), you need to write transformation code for expression trees. A lot of it.

To quote Frans Bouma, author of the O/R mapper LLBLGen Pro:

Writing a Linq provider is a lot of work which requires a lot of code. If you're dealing with a Linq provider which is just, say, 32KB in size, you can be sure it will not support the majority of situations you will run into. However, the O/R mapper developer likely simply said 'We have Linq support', and it's even likely the provider can handle the more basic examples of a single entity type fetch with a Where, an Order By or even a Group By. But in real life, once you as a developer have tasted the joy of writing compact, powerful queries using Linq, you will write queries with much more complexity than these Linq 101 examples. Will the Linq 'provider' you chose be able to handle these as well? In other words: is it a full Linq provider or, as some would say, a 'toy' ?

Of course, it’s very possible to write a full LINQ provider. You just need to invest a lot of time and a lot of brain. There are also great resources online, for example Matt Warren’s Wayward Weblog, which hosts a LINQ provider tutorial, which was eventually included by the MSDN library. But even with the help of such resources, implementing a LINQ provider is nowhere near to trivial.

Which is why, about two years ago, a few people here at rubicon (most notably Stefan Wenig) came up with the idea of building a reusable LINQ provider foundation. The idea, to put it short, was to implement as much of the generic LINQ stuff once and for all, in a reusable way, so that new LINQ providers could start from a much higher level. The longer version of this can be found in a whitepaper I’ve written about the topic.

Matthias Görtler and myself were tasked with implementing that foundation, called re-linq, and we did so, with a lot of interruptions, but steadily evolving the framework. Now, its architecture has achieved maturity, and we’re encouraging everyone to try it out.

So, are you planning to write a LINQ provider? Try re-linq – it’s open-source (LGPL) – and it will save you a lot of headaches.

In my next post, I’ll outline the steps you need to take to write a LINQ provider using re-linq.

- Fabian