pipe to chainable operators/functions

[weepy]

A lot of maths is based around binary operators e.g, * + - etc. They are really just a alternate function definition. E.g.

5 + 8 * 8 - 1    <==>   minus(plus(times(8,8),1),1)
or in general
a f b   <=>    f(a,b)

The main reason we tend to use f(a,b,c) type function definitions is because they can take an arbitrary number of arguments, rather than just the two of binary operators.

However as you can see they are much easier to parse or 'stream' due to the way they are set out. Ruby's enumerable functions tend to work like binary operators, which is why they are so nice to use (read chainable). This is also mimicked by libraries like jQuery and Underscore. Also the pipe operator in Unix works on a similar premise.

I've been wondering if it would be possible to provide a way for Coffee to provide operators that mapped to functions. I think it could provide a very nice way of programming.

This is really just a ticket to open a discussion about them, but in terms of syntax/implementation, there is an idea I had: to use the pipe operator to indicate the steam and rewrite the function flow. So for example:

map: (fn, A) -> fn(v) for v in A
detect: (fn, A) -> (return v if fn(v)) for v in A
                            
times: (x, y) -> x * y 
add: (x, y) -> x + y

y: 1  | add 8 | times 8
z: [1,2,3,4] | map (x) -> x*x | detect (x) -> x > 10

ok y is 72
ok z is 16

The piped lines would both be interpreted as:

y: times(8, add( 1, 8)) 
z: detect(((x) -> x > 10), map(((x) -> x*x), [1,2,3,4])

As you can see, the piped versions are much more readable :-D. Additionally using the pipe means that we're not restricted to two parameters; if we take something more general:

x | func1 args... | func2 args2.... | func3 args3...

This is mapped in reverse or 'inside-out' with the left hand pipe-in being appended as the last argument to the next function call. So this would end up something like:

func3( args3..., func2( args2..., func1( args..., c )))

Obviously we'd need to find an alternative for the current bitwise or |, but it could be quite powerful and help make much more readable code, as it flows it a way that's much more brain friendly

Thoughts ?

[StanAngeloff]

Lol, is it just the example or is it just me, but I much prefer the version without the pipe? In nix environments the pipe could be used to *pass data to another executable where the syntax here is inverted and much harder to understand.

Anyway, it is just my opinion -- I am sure someone else would find it rather useful and convenient.

[weepy]

I'm not sure why you think the syntax here is inverted. E.g

grep: (search, data) -> 
   ret: []
   for line in data
     ret.push(line) if line.match search
   ret

file.read("x").split("\n") | grep "cats"   # returns a list of lines containing the word "cats" 

[StanAngeloff]

Great, it was the example in the original post then! This one makes much more sense and is very practical.

[jashkenas]

Weepy -- This is pretty darn nice. But you still need to figure out how to pass the return value from the previous function into the next one -- is it the first argument, or is it the last argument? Or can you pass more than one argument along?

Also, it doesn't play nicely with OOP-style calls directly on the object.

array | (x) -> x * x | reverse

How would you call reverse on the array, given that it's not an external function?

If you have utilities, where each is stand-alone and stakes a single STDIN and writes to a single STDOUT, piping can be uniform. But with functions attached to objects with arbitrary arguments, it's more complicated.

[weepy]

is it the first argument, or is it the last argument

It can be either ... both have advantages, but it shouldn't matter too much either way. As far as "passing more arguments along" goes, I'm not quite sure - can the unix pipe do this ?

I think your example is missing a map utility? As far as doing OOP style calls onto the output, you could either wrap in parentheses:

(array | map (x) -> x * x).reverse()

or perhaps have a naked . directly after the pipe ?

array | map (x) -> x * x | .reverse()

It can trivially support more than one option on the 'utility' function:

array | sort_limit 5, "DESC"  | .reverse()

But it might be difficult to have more than one "input".

I'm not arguing that this is better or more applicable or a replacement for standard function calls. It does have limitations (which in fact lead directly to it's strength) and for particular parts of coding it could be useful, particularly for data manipulation.

[matehat]

That idea is awesome. But it does need to be sorted out.

I think a natural and intuitive rule for passing arguments to functions could be that the previous result be used as the first argument and the rest be added to it. This way the reverse example could read :

[0..5] | map (x) -> x*x | Array::reverse.apply

There could also be a convention to encounter the case where the previous operation did not return a value, like using the previous one or maybe add half-expression :

array | map (x) -> undefined | or alternate
array | map (x) -> x*x | Array::reverse.apply | or this

[weepy]

Ok - I think that we're agreed that the inbound argument should be used as the first argument to the function, so map would look something like:

map: (A, fn) -> fn(v,i) for v,i in A

As far as calling other calling conventions, how about this rule :

• if the next token after the pipe is an operator (. or etc) then wrap the whole of the left hand side in parentheses and call the operator.
• otherwise treat the token as a function.
• it would also be neat to be able to handle white space nicely

  x: A | map (x) -> x \* x
   | .reverse()
   | or [1,2,3]
   | detect (x) -> x > 10
  

  to =>

  x: detect( (((map (A, x) -> x \* x).reverse()) or [1,2,3]), (x) -> x > 10) 
  

[StanAngeloff]

Alrighy, here is the commit on my pipe branch.

1. This will not work:

x: A | map (x) -> x * x | B

map argument must be wrapped in parentheses:

x: A | map((x) -> x * x) | B

2. The pipe character cannot start a new line:

A
| B

The pipe character can be used to end a line however:

A |
B

3. The only operator supported so far is | or. I am not entirely convinced others would be needed. | or can be treated as a special case to have a fallback value passed to the chain if the preceding call returned false/null/0/etc.

Jeremy, what do you think? Simple enough, no? Can you help with issue 2). I am hitting conflicts in the grammar when using the o "Expression INDENT | Expression" in grammar.coffee[Pipe].

As for 1) I would not consider that an issue.

[weepy]

1. It would be nice to have without the ()'s
2. I think | . is necessary too ?

[matehat]

Wonderful work Stan

[weepy]

It might be worth aliasing the standard | bitwise or to another operator to avoid ambiguity ? (it's barely used anyhow)

[StanAngeloff]

weepy, the following would still produce the expected result:

a: 2 | 1
b: c() | a

The new pipe syntax kicks in only if we have an invocation to the right:

c: b | d()

I reckon as long as this is documented, it shouldn't be a big issue. I certainly don't do bitwise operations between method calls.

EDIT: the new pipe syntax could be really useful for something close to LINQ:

rows: from('tableName') |
      where('column1', 'value') |
      select 'column2'

and if we clean it up to have optional parentheses:

rows: from   'tableName' |
      where  'column1', 'value' |
      select 'column2'

[jashkenas]

Ok. Pipes were briefly on master this morning, and we had a spirited debate about them on #coffeescript. Despite their fundamental appeal, here are the reasons to leave them out:

• They only replace a subset of function calls. If the argument you'd like to pipe in isn't the first argument, you're out of luck.

• Almost all of the syntax they enable can be accomplished just as cleanly through method chaining. Stan's example in the previous comment could also be written as:

  rows: from('table')
    .where('column1', value)
    .select('column2')
  

• Some functions fundamentally take more than one "main" argument (operations, for example).

• Working with OOP-style method calls (instead of external functions) is made more awkward in the presence of pipes.

So, closing as a wontfix. If you'd like to try pipes, Stan's branch is a great implementation.

[weepy]

Ah -- unfortunately I missed the spirited discussion :-(!

The trouble with achieving the same through method chaining is that all the methods need to be placed on the objects themselves. This isn't very good for Object and Array as we all know.

I was hoping that it could lead to a nice data manipulation syntax for Coffee. It's limitations obviously mean that any functions that use it would have to be specifically designed so (first argument for the data), but I don't think that means they wouldn't be useful enough?
As I said before - I don't think that you'd want to be using pipes in every line of code - but I think they'd still be very useful.

[jashkenas]

If you already have to define special functions to use it, then you might as well design those special functions to enable chaining.

[weepy]

Yes you are right. I suppose the main thrust of what I'm trying to achieve is ruby's Enumerable for Javascript's built in classes, without breaking Array and Object prototype's. Underscore is a similar attempt and it's only downfall is the slightly cumbersome syntax. Since a large portion Coffee/Javascript involves this data manipulation - it seems a shame that we don't have a decent way to address it.

What's your position on this?

[jashkenas]

If you want to use Ruby-in-JavaScript, the best library for that has always been Prototype.js. I know that it doesn't work on the server-side, but if you're doing a browser project, Prototype should let you write your example from above:

array | map (x) -> x * x | .reverse()

As:

array.map((x) -> x * x).reverse()

[jashkenas]

There's also:

http://github.com/280north/narwhal/blob/master/engines/default/lib/global-es5.js

Which adds the ECMAScript5 utility functions in a safer fashion, including map, reduce, some, etc. Here's a sample CoffeeScript session using that (after removing the JSON bit):

coffee> require './global-es5'
{}
coffee> [1, 2, 3, 4, 5].map((x) -> x * x).reverse()
[ 25, 16, 9, 4, 1 ]

[weepy]

That's pretty nice - thanks J

[matehat]

Hm, that's a good one ..

[weepy]

Just looking at this code from there.

Array.prototype.forEach =  function(block, thisObject) {
        var len = this.length >>> 0;
        for (var i = 0; i < len; i++) {
            if (i in this) {
                block.call(thisObject, this[i], i, this);
            }
        }
    };

What is the >>> ? It seems to work a bit like ||

[zmthy]

Does some weird division stuff. x >>> 0 is like parseInt(x, 10) though.

[zmthy]

For positive numbers, anyway.

[StanAngeloff]

I'd like to argue that having pipes would allow us to write new code in a more expressive way. Python and negative slices are dead and having an extend/include keyword in the core is out of the question. While all the arguments against pipes had to do with existing code, here is something you might be able to write using them:

value: list | at -1
value: list | take 1, 20, 2  # every other

The alternative is obvious, but not as expressive.

left: | include right

We are now pretty much used to half-operators and the above should be a nice addition. Rubysim?

Order | belongs_to User

So my main argument again is: look at pipes as means for you to write code in a new way. Sure, you can do value: get list, -1 but isn't it more natural to say value: list | at -1?

[weepy]

i think it's one of those ideas that would need trying in real code before figuring out whether it's good or not (mainly cos it needs a bit of a Brain Backflip) :-)

[StanAngeloff]

Can we consider using an alternative syntax which should allow anyone using the | pipe with methods to continue to use them? I quite like the ~. It even points to the right which could further improve readability:

POOL_ALLOCATE_INTERVAL: 15 * 60 * 1000
# vs.
POOL_ALLOCATE_INTERVAL: 15~minutes

[StanAngeloff]

15 lines of diff to add it in the core?

[dbrans]

I know I'm late to the discussion.
What if the piped value became the 'this' in the next expression? Instead of

fn.call obj, arg1, arg2, arg3

You would write

obj | fn arg1, arg2, arg3

Would work nicely with anonymous functions:

get id | -> 
  @name = 'Watson'
  @age = 23

[jashkenas]

using = (obj, func) -> func.call obj

using id, ->
  @name = "Watson"
  @age = 23

[dbrans]

Jeremy, I'm not a big fan of 'using'.
In JavaScript, you often call/apply methods that are not attached to any 'this'.

Array::slice.call arguments, 0

Why don't we put an end to that completely?
In the general case:

method.call x, arg1, arg2

... could be

x | method arg1, arg2

... which is pretty close to

x.method arg1, arg2

[matehat]

I think this is a nice way to put it! It makes sense and would certainly be more readable than method.call all the time!

[joeytwiddle]

Hugh Jackson's Tap-chain allows us to write calls in the order they will be evaluated.

averageAge = people.map( (p) -> return p.age )
                       .reduce(add)
                       .tap(divide, people.length)
                       .tap Math.round

Parenthesis are needed for all but the last call.

https://npmjs.org/package/tap-chain

[vendethiel]

Like underscore does. For chaining, there's other issues opened