Effectful Juvix tree evaluator #2623

janmasrovira · 2024-02-06T10:38:39Z

This pr implements two additional versions of the Juvix Tree evaluator. Now we have

The raw implementation that does not use effects. It throws Haskell exceptions for errors. It uses unsafePerformIO for traces. It relies on bang patterns to force strictness and guarantee the expected order of execution (for traces). Avoiding effects allows for improved execution efficiency.
polysemy based implementation.
effectful-core based implementation.

One can specify which evaluator to use thus:

juvix dev tree eval --evaluator XXX test.jvt

where XXX is one of raw, polysemy, effectful.

Preliminary benchmarks

More thorough benchmarks should be run, but here are some preliminary results:

Test032 (Fibonacci 20)

I've adapted test032 so that it main is a single call to fibonacci of 20.

Command:

hyperfine --warmup 2 --runs 10 'juvix dev tree eval test032.jvt --evaluator polysemy' 'juvix dev tree eval test032.jvt --evaluator raw' 'juvix dev tree eval test032.jvt --evaluator e
ffectful'

Output:

Benchmark 1: juvix dev tree eval test032.jvt --evaluator polysemy
  Time (mean ± σ):      2.133 s ±  0.040 s    [User: 2.113 s, System: 0.016 s]
  Range (min … max):    2.088 s …  2.227 s    10 runs

Benchmark 2: juvix dev tree eval test032.jvt --evaluator raw
  Time (mean ± σ):     308.7 ms ±  13.8 ms    [User: 293.6 ms, System: 14.1 ms]
  Range (min … max):   286.5 ms … 330.1 ms    10 runs

Benchmark 3: juvix dev tree eval test032.jvt --evaluator effectful
  Time (mean ± σ):     366.0 ms ±   2.8 ms    [User: 345.4 ms, System: 19.4 ms]
  Range (min … max):   362.5 ms … 372.6 ms    10 runs

Summary
  juvix dev tree eval test032.jvt --evaluator raw ran
    1.19 ± 0.05 times faster than juvix dev tree eval test032.jvt --evaluator effectful
    6.91 ± 0.34 times faster than juvix dev tree eval test032.jvt --evaluator polysemy

Test034 (Higher-order function composition)

A modified version of test034 where main defined as call[exp](3, 12)

Command:

hyperfine --warmup 2 --runs 10 'juvix dev tree eval test034.jvt --evaluator polysemy' 'juvix dev tree eval test034.jvt --evaluator raw' 'juvix dev tree eval test034.jvt --evaluator effectful'

Output:

Benchmark 1: juvix dev tree eval test034.jvt --evaluator polysemy
  Time (mean ± σ):      7.025 s ±  0.184 s    [User: 6.518 s, System: 0.469 s]
  Range (min … max):    6.866 s …  7.327 s    10 runs

Benchmark 2: juvix dev tree eval test034.jvt --evaluator raw
  Time (mean ± σ):     835.6 ms ±   7.4 ms    [User: 757.2 ms, System: 75.9 ms]
  Range (min … max):   824.7 ms … 847.4 ms    10 runs

Benchmark 3: juvix dev tree eval test034.jvt --evaluator effectful
  Time (mean ± σ):      1.578 s ±  0.010 s    [User: 1.427 s, System: 0.143 s]
  Range (min … max):    1.563 s …  1.595 s    10 runs

Summary
  juvix dev tree eval test034.jvt --evaluator raw ran
    1.89 ± 0.02 times faster than juvix dev tree eval test034.jvt --evaluator effectful
    8.41 ± 0.23 times faster than juvix dev tree eval test034.jvt --evaluator polysemy

Test036 (Streams without memoization)

A modified version of test036 where main defined as call[nth](700, call[primes]())

Command:

hyperfine --warmup 2 --runs 5 'juvix dev tree eval test036.jvt --evaluator polysemy' 'juvix dev tree eval test036.jvt --evaluator raw' 'juvix dev tree eval test036.jvt --evaluator effectful'

Output:

Benchmark 1: juvix dev tree eval test036.jvt --evaluator polysemy
  Time (mean ± σ):      1.993 s ±  0.026 s    [User: 1.946 s, System: 0.043 s]
  Range (min … max):    1.969 s …  2.023 s    5 runs

Benchmark 2: juvix dev tree eval test036.jvt --evaluator raw
  Time (mean ± σ):     137.5 ms ±   7.1 ms    [User: 127.5 ms, System: 8.9 ms]
  Range (min … max):   132.8 ms … 149.8 ms    5 runs

Benchmark 3: juvix dev tree eval test036.jvt --evaluator effectful
  Time (mean ± σ):     329.0 ms ±   7.3 ms    [User: 289.3 ms, System: 37.4 ms]
  Range (min … max):   319.9 ms … 336.0 ms    5 runs

Summary
  juvix dev tree eval test036.jvt --evaluator raw ran
    2.39 ± 0.13 times faster than juvix dev tree eval test036.jvt --evaluator effectful
   14.50 ± 0.77 times faster than juvix dev tree eval test036.jvt --evaluator polysemy

lukaszcz · 2024-02-07T17:11:16Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+          v <- eval' _nodeSaveArg
+          withExtendedTemp v (eval' _nodeSaveBody)
+
+printValue :: InfoTable -> Value -> Text


There is code duplication here. Exactly the same function is available in Evaluator.hs.

lukaszcz · 2024-02-07T17:11:24Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+  ValString s -> s
+  v -> ppPrint tab v
+
+valueToNode :: Value -> Node


Code duplication

src/Juvix/Compiler/Tree/EvaluatorEff.hs

lukaszcz · 2024-02-07T17:12:45Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+    return val
+
+-- | Catch EvalError and convert it to TreeError.
+catchEvalErrorIO :: IO a -> IO (Either TreeError a)


Not needed if errors are not Haskell exceptions. And code duplication.

lukaszcz · 2024-02-07T17:12:54Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+    (\(ex :: EvalError) -> return (Left (toTreeError ex)))
+
+toTreeError :: EvalError -> TreeError
+toTreeError EvalError {..} =


Code duplication

lukaszcz · 2024-02-07T17:13:07Z

src/Juvix/Compiler/Tree/EvaluatorSem.hs

+          withExtendedTemp v (eval' _nodeSaveBody)
+
+printValue :: InfoTable -> Value -> Text
+printValue tab = \case


Code duplication

lukaszcz · 2024-02-07T17:13:14Z

src/Juvix/Compiler/Tree/EvaluatorSem.hs

+  v -> ppPrint tab v
+
+valueToNode :: Value -> Node
+valueToNode = \case


Code duplication

lukaszcz · 2024-02-07T17:13:40Z

src/Juvix/Compiler/Tree/EvaluatorSem.hs

+    return val
+
+-- | Catch EvalError and convert it to TreeError.
+catchEvalErrorIO :: IO a -> IO (Either TreeError a)


Duplicated and not needed for the effects-based version

lukaszcz · 2024-02-07T17:13:50Z

src/Juvix/Compiler/Tree/EvaluatorSem.hs

+    (\(ex :: EvalError) -> return (Left (toTreeError ex)))
+
+toTreeError :: EvalError -> TreeError
+toTreeError EvalError {..} =


Code duplication

lukaszcz · 2024-02-07T17:14:10Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+eval :: (Output Value :> r, E.Error EvalError :> r) => InfoTable -> Node -> Eff r Value
+eval tab = E.runReader emptyEvalCtx . eval'
+  where
+    -- eval' :: forall r'. (Members '[Output Value, Reader EvalCtx, Error EvalError] r') => Node -> Sem r' Value


Remove comment

lukaszcz · 2024-02-07T17:14:23Z

src/Juvix/Compiler/Tree/EvaluatorEff.hs

+  mapM_ handle l
+  pure a
+
+-- eval :: (Members '[Output Value, Error EvalError] r) => InfoTable -> Node -> Sem r Value


Remove unnecessary comment

lukaszcz

Avoid code duplication

janmasrovira self-assigned this Feb 6, 2024

janmasrovira force-pushed the sem-tree-evaluator branch 2 times, most recently from 989a8a2 to 33e14c3 Compare February 6, 2024 14:55

janmasrovira changed the title ~~Sem Juvix tree evaluator~~ Effectful Juvix tree evaluator Feb 6, 2024

janmasrovira assigned paulcadman Feb 6, 2024

janmasrovira force-pushed the sem-tree-evaluator branch 7 times, most recently from bcf822b to f52cebc Compare February 7, 2024 11:49

janmasrovira added the enhancement New feature or request label Feb 7, 2024

janmasrovira marked this pull request as ready for review February 7, 2024 13:58

janmasrovira requested a review from paulcadman February 7, 2024 13:59

janmasrovira force-pushed the sem-tree-evaluator branch from 7b6a458 to 01db9c3 Compare February 7, 2024 14:04

lukaszcz self-requested a review February 7, 2024 16:06

lukaszcz reviewed Feb 7, 2024

View reviewed changes

src/Juvix/Compiler/Tree/EvaluatorEff.hs Show resolved Hide resolved

lukaszcz reviewed Feb 7, 2024

View reviewed changes

lukaszcz requested changes Feb 7, 2024

View reviewed changes

janmasrovira force-pushed the sem-tree-evaluator branch from 01db9c3 to 1993ad2 Compare February 7, 2024 17:51

janmasrovira requested a review from lukaszcz February 7, 2024 17:54

janmasrovira added 11 commits February 8, 2024 08:31

implement monadic evaluator for Juvix Tree

5172c1b

add doEvalSem to app TreeEvaluator

6a7b1b7

evaluator without bangs

52215b6

add effectful

b83e767

implement the Juvix Tree evaluator using effectful

657b87c

remove EvaluatorBangs

f577710

add cli option to select evaluator

a75541b

rename module

a6b68e7

reduce code duplication

a5ea9fb

remove comments

e19befe

ormolu

37869e6

paulcadman force-pushed the sem-tree-evaluator branch from 991c1ea to 37869e6 Compare February 8, 2024 08:31

lukaszcz approved these changes Feb 8, 2024

View reviewed changes

lukaszcz merged commit 8dfe2ba into main Feb 8, 2024
4 checks passed

lukaszcz deleted the sem-tree-evaluator branch February 8, 2024 09:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Effectful Juvix tree evaluator #2623

Effectful Juvix tree evaluator #2623

janmasrovira commented Feb 6, 2024 •

edited

Loading

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz Feb 7, 2024

janmasrovira Feb 7, 2024

lukaszcz left a comment

Effectful Juvix tree evaluator #2623

Effectful Juvix tree evaluator #2623

Conversation

janmasrovira commented Feb 6, 2024 • edited Loading

Preliminary benchmarks

Test032 (Fibonacci 20)

Test034 (Higher-order function composition)

Test036 (Streams without memoization)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukaszcz left a comment

Choose a reason for hiding this comment

janmasrovira commented Feb 6, 2024 •

edited

Loading