Initial implementation of the Encore task library

[kikofernandez]

Tasks are implemented as actors, where each scheduler has a specific task runner (actor for tasks). Tasks returns futures; at the moment, you cannot chain two tasks together to create dependencies between them, but you can create a task and chain it to a function (typical chain support).

there are some couple of examples for tasks:

• global_async.enc, tests using async inside a global function
• async_force_gc.enc, tests async creating many futures and forcing gc on the object
• async_foreach.enc, tests async side effects, we don't block on the future produced by the task
• async_chain.enc, simple example of chaining a task to an actor

[kikofernandez]

@albertnetymk I have tested it under Vagrant but, could you test that it works on Linux?

[TheGrandmother]

I ran the test on my machine(Ubuntu 14.10) and it works!

And don't forget to add tasks to the documentation :)

[kikofernandez]

@TheGrandmother thanks! Doc added!

[TobiasWrigstad]

Will tasks ever block explicitly on futures? I mean will there be a call to "block" inside the code of a task?

[kikofernandez]

potentially, there could be. that case is not supported at the moment. what I mean is that there could be a case such that:

def process(a: Agent): Cat {
  let fut_cat = a.getCat() in
    get fut_cat
}

class Main
  def main(): void {
    ...
    let f = async(process(agent)) in 
      ...
  }

[albertnetymk]

Works OK on my Linux. One comment on the syntax: async(f()), seems to suggest that async is one function, but it doesn't follow the call convention of other functions, call-by-value, I believe. One better way could be to view it as a keyword, like asyn f().

[kikofernandez]

@albertnetymk it's a keyword, the parenthesis are placed there to improve readability! :)

[supercooldave]

I think the syntax should be
async { block }

[supercooldave]

@TobiasWrigstad regarding whether tasks will block. I see no reason why not.
But, get must have await semantics. That is, they must not block the actor that runs them.

[kikofernandez]

yes, exactly (@supercooldave last comment), which means that I'll have to fix await

[kikofernandez]

@supercooldave I have added documentation and an example with the requested syntax. E.g.

def square(x: int): int
  x * x

class Main
  def main(): void {
    repeat i <- 5
      async {
        let s = square(i) in
          print square(s)
      }
  }

[supercooldave]

Sweet!!
Next week we (with Daniel) can talk about compiling Party Types!

[kikofernandez]

I had to rebase on top of master and fix some conflicts. It would be great if someone with a Linux box could confirm that everything still works. For those who checked out this PR, you need to remove the local branch and pull again in order to get the new changes.

If someone is familiar with the new tests that @kaeluka created, it would be great if someone could confirm that what I see is normal:

await.out - differ: char 14, line 2
ERROR: test await failed with output:
vvvvvvvvvvvvvvvvvvvv
Before await
After await
17
While awaiting
^^^^^^^^^^^^^^^^^^^^
cmp: EOF on -
ERROR: test deadlock_yourself failed with output:
vvvvvvvvvvvvvvvvvvvv

^^^^^^^^^^^^^^^^^^^^
cmp: EOF on -
ERROR: test largestream failed with output:
vvvvvvvvvvvvvvvvvvvv

^^^^^^^^^^^^^^^^^^^^
parametricPrint_src/Pair.encore.c:26:111: warning: data argument not used by format string [-Wformat-extra-args]
  ..."(Expr.hs: type_to_printf_fstr not defined for a,Expr.hs: type_to_printf_fstr not defined for b)\n", _field...
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  ^
1 warning generated.
parametricPrint.out - differ: char 2, line 1
ERROR: test parametricPrint failed with output:
parametricPrint.out - differ: char 2, line 1                                                              [16/1998]
ERROR: test parametricPrint failed with output:
vvvvvvvvvvvvvvvvvvvv
(Expr.hs: type_to_printf_fstr not defined for a,Expr.hs: type_to_printf_fstr not defined for b)
^^^^^^^^^^^^^^^^^^^^
suspend.out - differ: char 16, line 2
ERROR: test suspend failed with output:
vvvvvvvvvvvvvvvvvvvv
Before suspend
After suspend
Interleaving!
^^^^^^^^^^^^^^^^^^^^
     Tests passed: 60/65
     Tests failed: 5/65
         await
         deadlock_yourself
         largestream
         parametricPrint
         suspend
make -C modules test
compiling 'Importer'
Importing module Lib from ./Lib.enc
compiling 'DeepImporter'
Importing module MidImport from ./MidImport.enc
Importing module DeepLib from ./DeepLib.enc
compiling 'FarImporter'
Importing module FarLib from lib/FarLib.enc
compiling 'DupImporter'
Warning: Module DupLib found in multiple places:
-- lib/DupLib.enc
-- lib/DupLib.enc
-- otherlib/DupLib.enc
compiling 'Qualified'
Importing module A.B.QLib from lib/A/B/QLib.enc

Module tests:
     Tests passed: 5/5
     Tests failed: 0/5

[albertnetymk]

On my Linux, the result is the same except async_block. Because tasks are run in parallel, we can't specify the execution order.

[kikofernandez]

@albertnetymk updated the test as requested :)

[supercooldave]

I've run this and it looks fine.
Any objections to merging it?

[albertnetymk]

If I understand it correctly, the logic goes like this: for each iteration, thread local task_runner picks one task_msg from global task_q, and call handle_msg on it.

Possible improvement:
Uses dedicate class instead of Main class to construct task_runner, like cycle_detector
Uses dedicate task_handler instead of handle_msg to handle tasks, for some logic in handle_msg doesn't make sense for tasks.

Other than that, it looks good as the first implementation.

[kikofernandez]

@albertnetymk yes, that's how it works.

First improvement: you are right, I took the main class because it's auto-generated and I need not change anything in the PonyRT (=> less conflicts when merging)

Second improvement: I do not agree, it's true that it could be optimized but I decided to treat every message in the same way. The handle_msg contains the logic for using the eager or lazy switch context and I would like the task runners to keep it in the same way. Otherwise we don't treat them as equal, when I believe the task runners do not need to concern with how to handle a message, they'll do whatever any other actor does (in the sense of lazy or eager).

Thanks for your feedback, it's always appreciated! 👍

[supercooldave]

Note that because this was open for so long, it now has conflicts with my recent commits -- I changed the AST a little.

[kikofernandez]

@supercooldave thanks, I just fixed it.
I have pushed the finish language construct in this PR as well.

[supercooldave]

I will check over this tonight. It's be good if @albertnetymk gives his approval too.

[supercooldave]

(I hope you have squashed all that needs to be squashed.)

[supercooldave]

Looks good.