Could we make iterator helpers more ergonomic for working with iterables?

[littledan]

In the Number.range proposal, iterator helpers are cited as a big reason why that API would vend an iterator, rather than an iterable tc39/proposal-iterator.range#17 . This makes me feel a bit uneasy; I thought the general JS convention for these things would be iterables. I wonder if there's something we could do in iterator helpers to make it more ergonomic to use them than requiring all uses to start with Iterator.from (which may be considered not ergonomic enough).

Here's an idea: For each iterator method, we make a static version for Iterator, e.g., Iterator.take would call Symbol.iterator on its first argument, and then pass the rest of the arguments on to the take method called on the result. So, if Number.range returns an iterable, then the examples could be written as follows (all other examples remain unchanged):

// With iterator helper proposal
Iterator.take(Number.range(0, Infinity), 1000)
    .filter(x => !(x % 3))
    .toArray()

[ljharb]

All iterators are iterable, by convention. Iterator.from is to produce an iterator from an iterable.

[ljharb]

In other words, if Number.range produces an iterator directly, then it'd already inherit from Iterator.prototype and have take and friends available directly on it.

[devsnek]

I'd be surprised if Number.range() didn't return an iterator (e.g. I'd expect Number.range().next() to be a thing). As Jordan says though, iterators are also iterable. Iterator.from exists to automatically call Symbol.iterator and also to wrap iterators that don't inherit from %Iterator.prototype%.

[littledan]

The only justification I've understood for why Number.range should return an iterator, and not a reusable iterator, was ergonomics for iterator helpers. If people see other reasons besides iterator helpers for this decision, it'd be good to follow up in tc39/proposal-iterator.range#17 (thanks for starting this already @devsnek in tc39/proposal-iterator.range#17 (comment))

I'm not sure if my initial comment was clear--I understand that the Number.range proposal currently produces an iterator. My thought was that it would make sense for it to make a (reusable) iterable instead, so I was thinking about how we could make that work without sacrificing overall ergonomics, if Iterator.from is considered too wordy.

[ljharb]

Number.range is the reusable iterator factory. What's the use case for having it produce another iterator factory with the arguments cached (eg, const factory = Number.range(x, y); [...factory(), ...factory()]) versus doing this: const factory = () => Number.range(x, y); [...factory(), ...factory()] or function* factory() { yield* Number.range(x, y); } ?

[littledan]

Let's keep this issue scoped to the topic of, how should iterator helpers work with iterables. Arrays should be a concrete enough example to work with, right? It's useful to deal with them in a lazy/streaming fashion sometimes, right? We can discuss Number.range in tc39/proposal-iterator.range#17

[devsnek]

@littledan the topic has been discussed ad nauseam in this repo, I'd encourage you to read over some of the closed issues on iterables (for example, #8, #68, #18)

[littledan]

Thanks for those cross-references. I haven't yet found any discussion in the issues or in the committee about leaving this proposal to be primarily about iterators, but also to improve the conveniences for working with iterables, beyond Iterator.from. I'm not arguing that this proposal should switch to being based on iterables.

To be clear, I'm definitely happy about the inclusion of Iterator.from, but I was disappointed by the comments in Number.range about that not being ergonomic enough. (There may be other reasons for Number.range to be an iterator rather than an iterable, I don't want to debate that here; I think we all agree that we'll want to use this API after starting with iterables sometimes, even if Number.range is an iterator.)

Do you see downsides to adding static methods to Iterator, for each iterator helper method, to call Symbol.iterator and then invoke the iterator helper method? (It could invoke the original iterator method, if no method is present, to better support iterators which don't subclass Iterator.prototype, as some people are raising in the other issues.) @rauschma previously suggested using a namespace object for iterable functions; this would amount to that while keeping this coherent with a iterator-based model.

[devsnek]

I don't see any benefit to adding those functions. I think they would only confuse people about iterable and iterator reusability.

[hax]

I have the same feeling with @littledan , basically it seems all iterables should have map/filter/take..., but I don't know how we can make it. Maybe first-class protocol proposal can help? I am not sure.

[littledan]

@hax I don't see how first-class protocols are connected. What do you mean?

[hax]

@littledan I mean, it seems the helpers should be iterable helpers instead of iterator helpers. But how to make a concrete class (especially those already exist classes --- which hard to change the prototype chain) get these iterable helpers? Maybe mixin or first-class protocol proposals (personally i prefer first-class protocol than mixin, but first-class protocol itself do not have ergonomic syntax yet, so I'm still thinking...)

[Jack-Works]

Umm some iterators don't inherited from Iterator.prototype and Iterables have their own iterator in general (a class that iterable)

Maybe Iterator.from is the only solution.. m

[ljharb]

I believe it is - “iterable” is an adjective, not a noun, by which i mean, it has a single interface that categorizes it as such - one that can not be enlarged. As such, iterables (or thenables, or toStringables or toJSONables etc) can’t be gifted extra methods, because they’re not an identity, they’re a shared trait.

Iterators, however, while they could be described the same (they have a next method) by convention all inherit from Iterator.prototype, which Iterator.from helps wrap things with.

[Jack-Works]

What if we let iterator helpers accept iterable as it's "this" value.

When "this" is not a iterator but iterable, it automatically call the Symbol.iterator.

Example:

class X extends Iterator {
// inherit all iterator helpers from the extends clause

[Symbol.iterator]: ....

}

new X().take()
// Implicitly call X.@@iterator

[devsnek]

This issue really seems to just be rehashing things from a year ago so I'm going to close it out.

[hax]

@devsnek I understand that this problem may have been discussed many times and you may already have a conclusion. But if a problem was raised again and again, it may be a signal that we may miss something, at least we need an open place to continue the discussion.

[hax]

“iterable” is an adjective, not a noun

@ljharb I don't know how adjective/noun have difference in this problem. For me, "iterable" are just objects implement Iterable interface, and "iterator" are objects implement Iterator interface.

by convention all inherit from Iterator.prototype

I think this is the only difference, that builtin iterators have "IteratorPrototype" as a "shared trait" installed.

The interesting part is, if we look at "IteratorPrototype", it only have [Symbol.iterator]() now. So "IteratorPrototype" actually is not Iterator (though it could be think as the abstract base class or shared trait of iterator).

If we have the convention that all iterators should inherit from IteratorPrototype (to be honest, I don't see any js articles/books stress this point before), then it means Iterator should be the subtype/subclass of Iterable.

So I think the suggested solution from @Jack-Works just works. Essentially it just change IteratorPrototype from the concept of the shared trait (or abstract base class) of Iterator to the shared trait (or abstract base class) of Iterable. Because iterators are also iterable, they of coz could "inherit" all helpers from the shared trait of Iterable.

The only left problem is can we change the name of "IteratorPrototype" to "IterablePrototype" or "prototypeForBothIterableAndIterator" to make it clear 😆

[ljharb]

The difference is that it's a simple transformation to change an iterator to one that inherits from Iterator.prototype; "iterables" include strings, maps, sets, arrays, etc, so it makes no sense to try to construct an inheritance hierarchy for all iterables.

[hax]

@ljharb I don't think it makes no sense. I mean, "construct an inheritance hierarchy for all iterables" sounds like crazy, but it's just the limitation of the language which only have single-root inheritance (include current JS), that's why I mention first-class protocol or mixin proposal. The ideal solution would be having a IterableProtocol (or IterableMixin) and IteratorProtocol (or IteratorMixin). But I think @Jack-Works 's solution is a workable solution as a tradeoff before we figure out how JS can have protocol or mixin in langauge.

[ljharb]

Given that it's the limitation of the language, it wouldn't make sense to design it that way - especially not if it depends on stage 1 proposals.

[hax]

I'm confused, @Jack-Works 's solution does not depend on stage 1 protocol/mixin proposals...

[ljharb]

I'm still talking about the "Iterables" approach.

@Jack-Works's solution is one I advocated for here, and it's what the spec already does in Iterator.from https://tc39.es/proposal-iterator-helpers/#sec-iterator.from.

Since all the prototype methods throw when the receiver lacks the [[Iterated]] internal slot, it would be a feasible follow-on proposal to relax that when the receiver was an iterable.

[hax]

and it's what the spec already does

Hope someone could add some explanation about such important semantic in README!

[Jack-Works]

Another thought:

Object.assign({}, obj);
( { ...obj } );

Array.from(iterable); // equal to
[...iterable];

Iterator.from(iterable); // equal to
// ??? The missing syntax

Add a new syntax to convert an Iterable to Iterator will relax the embarrassment situation with Iterator Helpers and Iterables / Iterator that not inherit from the Iterator.prototype.

I don't know what the syntax should be like but this may be a solution to this problem.