Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AlternativeFlatten #1337

Closed
wants to merge 5 commits into from
Closed

Add AlternativeFlatten #1337

wants to merge 5 commits into from

Conversation

johnynek
Copy link
Contributor

@johnynek johnynek commented Aug 28, 2016

AlternativeFlatten gives us mapFlatten with a natural functor like law. Here is an example:

fa.mapFlatten { a => List(a, a) }.mapFlatten { b => Some(b) } == fa.mapFlatten { a => List(a, a).map(Some(_)) }

This is useful to model the kind of "flatMap" scala collections have, where we are not really dealing with Monadic flatMap. Also this is very useful for distributed systems such as spark and scalding, which offer this kind of abstraction but the core data type is not a monad (RDD and TypedPipe respectively).

I was talking about something like this, and @non suggested we use G[_]: Foldable as the inner type, upon trying to formalize the laws, I moved to G[_]: Traverse which seems to satisfy all use cases and allows us to state some nice laws.

@codecov-io
Copy link

codecov-io commented Aug 28, 2016

Codecov Report

Merging #1337 into master will increase coverage by 0.08%.
The diff coverage is 94.23%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1337      +/-   ##
==========================================
+ Coverage   91.39%   91.47%   +0.08%     
==========================================
  Files         237      241       +4     
  Lines        3567     3615      +48     
  Branches       64      120      +56     
==========================================
+ Hits         3260     3307      +47     
- Misses        307      308       +1
Impacted Files Coverage Δ
...rc/main/scala/cats/syntax/alternativeFlatten.scala 0% <0%> (ø)
...cats/laws/discipline/AlternativeFlattenTests.scala 100% <100%> (ø)
core/src/main/scala/cats/instances/stream.scala 95.74% <100%> (+2.72%) ⬆️
.../main/scala/cats/laws/AlternativeFlattenLaws.scala 100% <100%> (ø)
core/src/main/scala/cats/AlternativeFlatten.scala 100% <100%> (ø)
core/src/main/scala/cats/instances/list.scala 100% <100%> (ø) ⬆️
core/src/main/scala/cats/instances/vector.scala 97.87% <100%> (+0.5%) ⬆️
core/src/main/scala/cats/Functor.scala 52% <0%> (-48%) ⬇️
core/src/main/scala/cats/functor/Profunctor.scala 60% <0%> (-40%) ⬇️
core/src/main/scala/cats/arrow/Split.scala 66.66% <0%> (-33.34%) ⬇️
... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41d9ef8...8f9315c. Read the comment docs.

@kcsongor
Copy link
Contributor

I don't think this makes sense as a separate type-class.
I would love for someone to show me how FunctorFilter can be implemented for Functors that aren't equpped with a monoidal property, (i.e. that aren't Applicative).
In case it can't, then I think this is duplicating what Traverse does?

@kcsongor
Copy link
Contributor

actually not applicative, but something with a f a -> f a -> f a

@johnynek
Copy link
Contributor Author

I'm a bit confused. Can you show a proof that this is equivalent to
Traverse? I don't see it.

As I said, scalding and spark (not to mention the scala standard library's
flatMap) are all really this mapFlatten operation I'm proposing.

In the absence of a proof of equivalence, I'd say it is clearly general.
On Mon, Aug 29, 2016 at 09:36 Csongor Kiss notifications@github.com wrote:

actually not applicative, but something with a f a -> f a -> f a


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdsoJPS4hpcCWoJUkfOJhaLNSKPgZks5qkzSlgaJpZM4Ju0KK
.

@ceedubs
Copy link
Contributor

ceedubs commented Aug 29, 2016

I could be wrong about this, but after taking a quick glance it seems to me that if we implemented an instance of this type class that only ever did anything with the first item in the provided G (and therefore we could create an instance for Option) it would satisfy the laws, but to me this seems like it would violate unwritten assumptions about what this type class represents. Is there a way that these assumptions can be codified as laws? I'm not sure if there's a way you could unless you coupled this with something like Unfoldable.

@johnynek
Copy link
Contributor Author

I think if you have MonoidK and Applicative on F[_] you can make a law like:

pure(a).mapFlatten(f) == f(a).foldRight(empty) { (b, end) =>
combineK(pure(b), end) }

On Mon, Aug 29, 2016 at 09:58 Cody Allen notifications@github.com wrote:

I could be wrong about this, but after taking a quick glance it seems to
me that if we implemented an instance of this type class that only ever did
anything with the first item in the provided G (and therefore we could
create an instance for Option) it would satisfy the laws, but to me this
seems like it would violate unwritten assumptions about what this type
class represents. Is there a way that these assumptions can be codified as
laws? I'm not sure if there's a way you could unless you coupled this with
something like Unfoldable #1132.


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdpVaARjUZVg6wgGcDN3TsmENvB6iks5qkznNgaJpZM4Ju0KK
.

@kcsongor
Copy link
Contributor

kcsongor commented Sep 9, 2016

@johnynek sorry for the delay, I have just gotten around to writing down a few thoughts I've had on this: #1365
However, I don't have a full proof, it is difficult to prove the 'necessary' part. I think the question really is whether it is possible to remove an element from a structure that doesn't have a foldable instance (by doesn't have, I mean it cannot admit a lawful instance for some reason).
I'll try and come up with a proof for algebraic data types, and then think about the general case, or prove myself wrong.

@non
Copy link
Contributor

non commented Sep 9, 2016

@kcsongor So to help motivate the type class, consider TypedPipe[A].

This type is a bit like a Task[A] but also like a collection. It represents operations on zero-or-more rows of data in a distributed filesystem which you can't easily access, and which won't be run until the end of the world.

It definitely has a Functor instance (it has a standard map method). Since it also has a pure method and the ability to create products (in its case Cartesian products), its instance is Applicative. It also has universal concatenation and empty methods, so it has a MonoidK instance as well, making the whole structure an Alternative instance.

However it doesn't a have true flatMap method (i.e. A => TypedPipe[B]); instead it supports something like A => Iterable[B] (or A => G[FB] for Foldable[G]). You can't actually extract A values out of TypedPipe[A], so it itself is not foldable.

It could be that there aren't any FunctorFilter instances which aren't instances of this more powerful type class. I agree with your reasoning that FunctorFilter (and this type class) should require empty since you can derive it using .filter(_ => false). I can't easily think of types where you can contract (but not expand) their structure.

However I think it's worth considering this type class on its own merits. Once you have a full-blown Monad you give up a lot of control around its execution (compared to an applicative structure). Whether we call this FunctorFlatten or instead create some kind of souped-up version of Applicative or Alternative I feel like this would be a useful addition.

@ceedubs ceedubs mentioned this pull request Sep 9, 2016
@kcsongor
Copy link
Contributor

kcsongor commented Sep 9, 2016

@non Thanks for that example, it was very useful!

The assumption I made was that the intermediate values need to be of the same type as the container, but that assumption was too restrictive, since it is enough that we can produce some intermediate values of type G[B], which can be collapsed into a single G[B], which can then be turned back into an F[B].
So, as long as there is a natural transformation G ~> F, I can weaken the restriction from flatMap[A, B](fa: F[A])(afb: A => F[B]) => F[B] to flatMap2[A, B, G[_]: Foldable](fa: F[A])(afb: A => G[B]) => F[B]
This is still coming from FunctorFilter, but flatMap2 now looks exactly like mapFlatten from FunctorFlatten!

I understand that this is useful from a practical perspective, when you can write a more efficient way of filtering, and don't want to use some almost incidental filter function that is derived from other classes.

@andyscott
Copy link
Contributor

From a learning perspective, someone who is familiar with Scala and Scala's flatMap may find the principled flatMap irritating or too rigid to work with. Providing a mapFlatten or something similar (that's easily accessible) would ease the adoption process. I would find it convenient in my own work as well.

@johnynek
Copy link
Contributor Author

I'll try to rework this, perhaps based on Alternative (all the examples I
know of are also Alternative).
On Sat, Sep 17, 2016 at 07:01 Andy notifications@github.com wrote:

From a learning perspective, someone who is familiar with Scala and
Scala's flatMap may find the principled flatMap irritating or too rigid to
work with. Providing a mapFlatten or something similar (that's easily
accessible) would ease the adoption process. I would find it convenient in
my own work as well.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdh9uD6XUvAYVwSBYDsu6iR-19NNMks5qrBzbgaJpZM4Ju0KK
.

@adelbertc
Copy link
Contributor

adelbertc commented Sep 18, 2016

As a side note, such existence of these type classes that branch off the "main" (I don't like that adjective since it implies theres a certain blessed type class hierarchy but I can't think of a better word right now) type class hierarchy can cause issues. For instance, if you have

def foo[F[_]: FunctorFlatten: Applicative] = ...

If you try to use anything that requires an implicit Functor[F] you will get ambiguous implicits. This is a manifestation of #1210 - more discussion here in this PR here: #1379

@johnynek
Copy link
Contributor Author

@ceedubs what do you think of the new laws? They seem legit to me?

@johnynek johnynek changed the title Add FunctorFlatten Add AlternativeFlatten Sep 20, 2016
Copy link
Contributor

@adelbertc adelbertc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see #1379 (comment) I would like to make a decision on that before adding more of such type classes like this.

@kailuowang
Copy link
Contributor

@johnynek are we still interested in this?

@sellout
Copy link
Contributor

sellout commented Mar 10, 2018

This should be in cats-mtl now, no? (Considering it extends classes that I think have moved to cats-mtl. Also, this should be a superclass of MonadCombine, maybe? (or whatever the equivalent in cats-mtl is now – I’ve fallen behind in this stuff)

@johnynek
Copy link
Contributor Author

@sellout the examples I care most about are not Monads.

@sellout
Copy link
Contributor

sellout commented Mar 10, 2018

@johnynek Right – I’m just wondering where this class fits in the hierarchy – is it between Alternative and MonadCombine, or is it a separate branch off Alternative? It seems like the former to me, since the fundamental flatten operation works with either (AlternativeFlatten m, Traversable f) => m (f a) -> m a or (MonadCombine m, Foldable f) => m (f a) -> m a, but that’s just a hunch.

@kailuowang
Copy link
Contributor

Closing stale PRs. Feel free to reopen if there is interest to revive the effort.

@johnynek
Copy link
Contributor Author

johnynek commented Nov 2, 2020

Note, SelectiveZero seems to be equivalent to this when G=Option:

https://github.com/snowleopard/ideas/blob/7692eba70758a48cf2dc9224627df906fe078a9b/src/SelectiveZero.hs#L41

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.