Issue #1316: Add the MonadDefer type-class #1552
Conversation
alexandru
changed the title
|
I will dive into the code later, but meanwhile, do you think adding a brief doc here https://github.com/typelevel/cats/tree/master/docs/src/main/tut/typeclasses with some example use cases to demonstrate the motivation would be helpful? |
|
@alexandru will you be at NEScala? I'd like to get a group together to discuss this. I don't think it should be part of cats because fs2 has its own and scalaz doesn't have one. Would be nice for this to be its own thing so we can all use it once and for all. |
|
@tpolecat FS2 has its own Monad too and this Yes, I'll probably be at NEScala - tried to submit a talk too, but some fine folks came ahead of me :-) |
| * | ||
| * Alias for `eval(always(a))`. | ||
| */ | ||
| final def delay[A](a: => A): F[A] = eval(always(a)) |
There was a problem hiding this comment.
should this be final? Many types, have their own implementation of this.
There was a problem hiding this comment.
eval and delay are redundant, imo we should have either one or the other. I added delay basically for familiarity purposes.
There was a problem hiding this comment.
yeah, but it is a shame to have to box to Eval just to get a method Task already has.
There was a problem hiding this comment.
I would have preferred eval to be described in terms of delay. This can go either way.
| * Lifts any value into the `F[_]` applicative context, where the | ||
| * evaluation is controlled by [[Eval]] and can be optionally lazy. | ||
| */ | ||
| def eval[A](a: Eval[A]): F[A] |
There was a problem hiding this comment.
would pure(a).map(_.value) be a legit implementation of this? If not, can you add a comment to help people understand the purpose of the type.
If this is not legit, can we point to exactly what law rules it out?
There was a problem hiding this comment.
If there's consensus, sure, though I like eval because it's just one verb.
|
|
||
| trait ApplicativeEvalLaws[F[_]] extends ApplicativeLaws[F] { | ||
| implicit override def F: ApplicativeEval[F] | ||
|
|
There was a problem hiding this comment.
don't we need something like:
var i = 0
val asF = F.eval(later { i += 1; i })
i == 0 // we have not incremented i yet.There was a problem hiding this comment.
No, because the purpose of ApplicativeEval is evaluation in the F[_] context, but it doesn't necessarily have to be lazy, such that Try and Future can implement it.
What you're thinking of is the other type class called MonadDefer.
| val lh = F.delay(tr(state1)) | ||
| val rh = F.pure(state2).map(tr) | ||
|
|
||
| lh.flatMap(_ => lh) <-> rh.flatMap(_ => rh) |
There was a problem hiding this comment.
can you comment on the intent of running these effects each twice and comparing them? I'd rather have an effectful thing on one side, and a non-effectful thing on the other so I can more easily read intent.
There was a problem hiding this comment.
The triggered code is side-effectul and thought it would be a good idea to test that delay(a) is equivalent with pure.map(_ => a) in the presence of side-effects. This doesn't necessarily have to be so. Consider ...
implicit object EvalInstances extends MonadDefer[Eval] {
def eval[A](a: Eval[A]) = a.memoize
}And now this law is violated.
There was a problem hiding this comment.
maybe it helps to add negative tests to be clear exactly what we are ruling out.
| @@ -25,6 +24,9 @@ class TryTests extends CatsSuite { | |||
| checkAll("Try", MonadTests[Try].monad[Int, Int, Int]) | |||
| checkAll("Monad[Try]", SerializableTests.serializable(Monad[Try])) | |||
|
|
|||
| checkAll("Try[Int]", ApplicativeEvalTests[Try].applicativeEvalWithError[Int, Int, Int]) | |||
| checkAll("ApplicativeEval[Try]", SerializableTests.serializable(ApplicativeEval[Try])) | |||
There was a problem hiding this comment.
can we add a negative test to show that ApplicativeEval[Try] and MonadDefer[Try] will fail the laws?
I think Try should not be lawful if this typeclass is really representing anything. Since any Applicative can do it if Try can, no?
There was a problem hiding this comment.
I also added this law (notice applicativeEvalWithError):
eval(always(throw ex)) == raiseError(ex)It's an optional law that happens in case F[_] also has an ApplicativeError[A, Throwable]. Mentioning this because Try follows this law, whereas Eval does not.
|
@alexandru but fs2 is not going to add a cats dependency, which means there will need to be another shim to support this. It's propagating the exact problem it's trying to solve. |
|
@tpolecat I think the problem here is that people want to support both Cats and Scalaz, in order to not upset anybody, I have the same problem in @monix, including my own I think it should be in Cats also because it needs But anyway, we can further discuss this at NEScala. I wanted to do this PR for quite some time and finally got it out of my system :-) |
Codecov Report
@@ Coverage Diff @@
## master #1552 +/- ##
==========================================
- Coverage 92.34% 92.33% -0.02%
==========================================
Files 247 250 +3
Lines 3907 3940 +33
Branches 132 140 +8
==========================================
+ Hits 3608 3638 +30
- Misses 299 302 +3
Continue to review full report at Codecov.
|
|
👍 goddamn it, finally. Maybe this means stack-safety work can continue. |
| * applicative context, but without the repeating side-effects | ||
| * requirement. | ||
| */ | ||
| @typeclass trait MonadDefer[F[_]] extends Monad[F] with ApplicativeEval[F] { |
There was a problem hiding this comment.
Is it worth providing a default instance of MonadDefer implemented as pure(()).map(_ => a) and pure(()).flatMap(_ => fa) for any Monad M? This way we could have a MonadDefer instance even for Id which can be useful.
There was a problem hiding this comment.
So I've added laws that eval should indeed be equivalent with pure(()).map(_ => a) and defer should be equivalent with pure(()).flatMap(_ => fa).
However Id cannot implement MonadDefer, because for this type-class we are specifying that any side effects should be repeated on each evaluation, with defer building a factory of F[A].
This means that MonadDefer can be implemented for cats.Eval, Task, IO, Coeval, Observable or other monads that handle side-effects, but cannot be implemented for Id. It would fail the specified laws - tried to come up with something meaningful there, would appreciate feedback on those.
Id can implement ApplicativeEval, which doesn't have such a restriction. Try and Future are other candidates for ApplicativeEval and that can't implement MonadDefer. I find MonadDefer cool because it describes laziness.
There was a problem hiding this comment.
I agree on Id not having a MonadDefer instance. It wouldn't be really lazy and stackoverflow errors would happen (just got one while trying to use Id as my "simplest MonadDefer monad").
|
I'm a bit confused: what Applicative can't implement ApplicativeEval? Why
not just add this method to Applicative?
…On Thu, Mar 9, 2017 at 20:51 Alexandru Nedelcu ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In core/src/main/scala/cats/MonadDefer.scala
<#1552 (comment)>:
> @@ -0,0 +1,21 @@
+package cats
+
+import simulacrum.typeclass
+import cats.Eval.always
+
+/**
+ * A [[Monad monad]] that allows for arbitrarily delaying the
+ * evaluation of an operation, triggering its execution on each run.
+ *
+ * @see [[ApplicativeEval]] for capturing effects in an `F[_]`
+ * applicative context, but without the repeating side-effects
+ * requirement.
+ */
***@***.*** trait MonadDefer[F[_]] extends Monad[F] with ApplicativeEval[F] {
So I've added laws that eval should indeed be equivalent with pure(()).map(_
=> a) and defer should be equivalent with pure(()).flatMap(_ => fa).
However Id cannot implement MonadDefer, because for this type-class we
are specifying that any *side effects* should be *repeated on each
evaluation*, with defer building a factory of F[A].
This means that MonadDefer can be implemented for cats.Eval, Task, IO,
Coeval, Observable or other monads that handle side-effects, but cannot
be implemented for Id. It would fail the specified laws - tried to come
up with something meaningful there, would appreciate feedback on those.
Id can implement ApplicativeEval, which doesn't have such a restriction.
Try and Future are other candidates for ApplicativeEval and that can't
implement MonadDefer. I find MonadDefer cool because it describes
laziness.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdnCGvrgLpvrVY3vm85QSwWrfXztbks5rkPLhgaJpZM4MYWn_>
.
|
|
@johnynek because you took it out :-) See: #1150 Indeed, any Note that not any |
|
Indeed, but the argument was that we couldn't find a law other than
something you can already get without it, but the Eval seemed to imply
something different.
Why should the Applicative function not be lazy, but Monad be lazy?
We definitely shouldn't add a typeclass if everything can satisfy it,
otherwise literally no one should only implement Applicative, right?
…On Thu, Mar 9, 2017 at 21:00 Alexandru Nedelcu ***@***.***> wrote:
@johnynek <https://github.com/johnynek> because you took it out :-) See:
#1150 <#1150>
Indeed, any Applicative can be an ApplicativeEval, although I've added an
*optional* law for ApplicativeError that says: eval(always(throw ex)) <->
raiseError(ex)
Note that not any Monad can be a MonadDefer. This one specifies laziness.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdsImkqXtL-pNxBlFjGcRCN8Q5KCzks5rkPTzgaJpZM4MYWn_>
.
|
|
To be clear: if we want to add a synonym for pure(()).map(_ => a) I guess
that's fine, but there seemed to be less agreement last time around that we
knew and agreed what these functions meant (specifically with regard to
when a side effect may be observed).
…On Thu, Mar 9, 2017 at 21:04 P. Oscar Boykin ***@***.***> wrote:
Indeed, but the argument was that we couldn't find a law other than
something you can already get without it, but the Eval seemed to imply
something different.
Why should the Applicative function not be lazy, but Monad be lazy?
We definitely shouldn't add a typeclass if everything can satisfy it,
otherwise literally no one should only implement Applicative, right?
On Thu, Mar 9, 2017 at 21:00 Alexandru Nedelcu ***@***.***>
wrote:
@johnynek <https://github.com/johnynek> because you took it out :-) See:
#1150 <#1150>
Indeed, any Applicative can be an ApplicativeEval, although I've added an
*optional* law for ApplicativeError that says: eval(always(throw ex)) <->
raiseError(ex)
Note that not any Monad can be a MonadDefer. This one specifies laziness.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdsImkqXtL-pNxBlFjGcRCN8Q5KCzks5rkPTzgaJpZM4MYWn_>
.
|
I can think of two reasons:
It's not a perfect synonym for This type-class specifies that if there is an |
|
Note that in FS2, we optimize
In FS2, we solve this by implementing |
|
Okay, now I see more clearly. Can we separate into two PRs possibly?
I think we are actually talking about two or more concerns here: a. a typeclass about exception capture into a monad (more than ApplicativeError since we want some laws around |
It's not.
What do they have in common? I can see 2 things:
We had this in And here I must mention that the Scalaz equivalent Applicative.point takes a by-name parameter and so it doesn't have this problem. In other words, in Scalaz we can abstract over the above As for |
|
My concern is that these four things are not similar in any way except that they take a by-name parameter.
So, if we have a typeclass, the idea, to me, is that it has some usable property that I only need to depend on. I don't see what it is here since there are several things it could be:
I think we should be clear what we want. But right now we seem to be saying, I want to unify the above 4 properties such that at least one of them is true. I don't see how I can usefully program with that. |
|
@johnynek Yes, they take a by-name parameter AND they are consistent with If you're looking that much into it, you might reach the same concerns about At this point I think we should talk use-cases. Here's the kind of code I'm dealing with right now: sealed abstract class Iterant[F[_], +A] { self =>
// ..
final def foldLeftL[S](seed: => S)(op: (S,A) => S)(implicit F: Monad[F]): F[S] = {
def loop(self: Iterant[F,A], state: S): F[S] = {
try self match {
case Next(a, rest, stop) =>
// Side-effects and exceptions possible here!!!
val newState = op(state, a)
rest.flatMap(loop(_, newState))
case Halt(None) =>
F.pure(state)
case Halt(Some(ex)) =>
throw ex
}
catch {
case NonFatal(ex) =>
// Closing resources!
earlyStop.flatMap(_ => (throw ex) : F[S])
}
}
val init = F.eval {
// Exception handling for the seed
try seed catch {
case NonFatal(ex) =>
earlyStop.flatMap(_ => (throw ex) : F[S])
}
}
init.flatMap(loop(self, _))
}
}For this piece of code I don't want to work with The whole point of higher-kinded polymorphism, the way I see it, is the ability to plug-in an So how can this operation be described with the current Cats type-classes and what should the solution look like? |
|
Thank you for taking the time to make a concrete example. I think it helps not talk past one another. First, Functor and Monad have well defined laws (composition of map and associativity of bind, etc...). While it is true some have lazy semantics, you get no promises and can't rely on it. To me, it looks like you do want MonadError since you explicitly want to handle the possibility of non fatal exceptions in java/scala code. You also appear to want an "always"-like pure, but maybe you don't care if it is always, but could actually be Now. Can you explain why you don't want MonadError? You are reimplementing catchNonFatal nearly exactly here. Lastly, I suspect you may say you don't want MonadError because you want to support Eval, but as your code demonstrates you can nearly hack Eval to make it support MonadError. What is missing is recover. I think we can add a recover node to Eval without hurting performance or stack safety of existing code. As you point out, on the JVM an exception may come at any time, so currently evaluating Eval may throw. It would be nice to have a method like: Eval.handleErrorWith[A](e: Eval[A])(fn: Throwable => Eval[A]): Eval[A]If Eval supported MonadError, would you feel that MonadError covered your use case? By the way, iteratee.io has many very similar cases and seems to use MonadError without it being a problem. I think there is still the separate suspend use case which must be lazy (or maybe have always semantics). That seems like something we don't have currently. But I still don't see a new typeclass that includes Try, Eval and Task other than Applicative. Maybe we could have a weaker thing than ApplicativeError where you can raise but not handle errors. Like ApplicativeRaise. Eval and Try can both be ApplicativeRaise with an error type of Throwable, it's just that Eval can't (yet?) handle the errors. I think we could maybe make a |
|
The reason that |
|
To be clear, I am all for MonadDefer as long as we can be clear about the laws. There seem to be at least two threads:
I don't know if both of those cases should be covered by a single typeclass. |
|
@johnynek My position on this is that we should provide a separate side-effect capture typeclass, potentially in a different PR. I don't think it necessarily needs any superclasses. What we will provide to users there is purely an abstraction that allows people to easily switch side-effect types, by pushing the decision to the call site. This is invaluable for libraries performing side effects that should not need to provide clones of their methods for different cats-compatible effect-capture libraries. I think the only law that is necessarily useful there is that whatever is suspended is not evaluated eagerly. |
|
Hi guys, So the current consensus is that we need If this is so, then I'll create another PR with just |
|
The history of this issue is interesting, for now I have not created a new PR, just updated this one. Changes:
Please re-review. Thanks! |
alexandru
changed the title
|
Can Free and FreeT implement this? It seems like they could. Also I think EitherT can if the F there can implement it (maybe any transformer can, ReaderT, WriterT, ... ) |
|
@johnynek Free and FreeT can only implement this if |
|
@edmundnoble I don't follow why Free.suspend is not a valid defer. What am I missing? |
|
@johnynek try Free[Id, A]. Then suspend does nothing for stack safety. |
|
Wait, why not? Free is trampolined. How is stack unsafe? Id has a good
tailRecM so running the Free is also safe. I don't see the problem.
…On Sat, Mar 25, 2017 at 12:34 Edmund Noble ***@***.***> wrote:
@johnynek <https://github.com/johnynek> try Free[Id, A]. Then suspend
does nothing for stack safety.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdmnx4UzltH-B93cD15vpN1dDTh77ks5rpZaCgaJpZM4MYWn_>
.
|
|
In fact Free[Id, ?] should be equivalent (but presumably less efficient) to
Eval, unless I'm missing something.
On Sat, Mar 25, 2017 at 12:36 P. Oscar Boykin <oscar.boykin@gmail.com>
wrote:
… Wait, why not? Free is trampolined. How is stack unsafe? Id has a good
tailRecM so running the Free is also safe. I don't see the problem.
On Sat, Mar 25, 2017 at 12:34 Edmund Noble ***@***.***>
wrote:
@johnynek <https://github.com/johnynek> try Free[Id, A]. Then suspend
does nothing for stack safety.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdmnx4UzltH-B93cD15vpN1dDTh77ks5rpZaCgaJpZM4MYWn_>
.
|
|
Okay. I recall something around this before. So after one evaluation it
will not be repeated with Free.
Is this also the case with FreeT do you know off hand.
…On Sat, Mar 25, 2017 at 12:58 Michael Pilquist ***@***.***> wrote:
@johnynek <https://github.com/johnynek> Re: Free.suspend and MonadDefer
-- #1277 <#1277>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1552 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEJdotR38l7vbw9OTuVzZFq5yBUm9CIks5rpZwBgaJpZM4MYWn_>
.
|
|
I haven't used |
|
@johnynek btw, the current definition for def catchNonFatal[A](a: => A)(implicit ev: Throwable <:< E): F[A] =
try pure(a)
catch {
case NonFatal(e) => raiseError(e)
}And in the instances defined in Cats, the only override I'm seeing is in
I remember mentioning this before, can't find the comment, but Cats is the only one with the problem because all the others have lazy behavior in |
|
I have included a Deferrable type-class and an Evaluable type-class in my proposal for the Typelevel Schrodinger project. This addresses two concerns:
If everybody agrees, I would like to close this pull request and this new project instead. |
|
Hello, I feel this issue has reached a deadlock. I don't want this to stop progress on alternative proposals, therefore I'm closing it. |
This pull request is about issue #1316. We are adding one new type class:
MonadDefer: for monadic types which are necessarily for lazy, as it repeats side-effects on each evaluation and have a stack safeflatMap. Implementing types:cats.Eval,IO,Task, etc.This is needed for abstracting over applicatives and monads that can capture effects (e.g.
Eval,Task) and libraries are doing their own thing:pure(()).map(_ => a)andpure(()).flatMap(_ => fa)and hope for the bestPlease checkout the described laws.
UPDATE: Originally this PR was introducing a second type class which was removed, leaving the description here for historical purposes:
ApplicativeEval, which would add anevaloperation that lifts values and evaluates side-effects in theF[_]context and is optionally lazy. This type-class does not require implementing types to be lazy, therefore it can be implemented for eager evaluation as well:Try,Future, etc.See the below discussion for why
ApplicativeEvalwas removed.