JEP Collections

In SDK we face several different collection types that we need to iterate, filter, map and reduce. While there are well established standards for doing these operations on JS arrays there are none for other types of collections, ideally we should have common API for all collections to make code more maintainable and easy to grok.

Scope

• Many XPCOM APIs return instances of nsISimpleEnumerator that come with special enumeration API:

  while (enumerator.hasMoreElements()) {
    process(enumerator.getNext());
  }

• Some code uses Mozilla specific, non standard features like iterators & generators that have special iteration semantics:

  // Mozilla JS
  for each (let value in generator()) {
    process(value);
  }

  Quite often nsISimpleEnumerator are wrapped into generators to present more JS-y API.

• JS Array' elements can be processed in variety of ways, both standard an non-standard:

  // Standard JS
  for (var i = 0, l = array.length; i < l; i++) {
    process(array[i]);
  }
  
  var i = 0, l = array.length;
  while (i < l) {
    process(array[i]);
  }
  
  // Mozilla JS
  for each (let value in array) {
    process(value);
  }

  There are also standard Array methods to these job:

  // Standard JS
  array.
    filter(predicate).
    map(f).
    reduce(accumulate);

• JS objects used as hash maps.

  // Standard JS
  for (var key in hash) {
     let value = hash[key];
     process(value);
  }
  
  // Mozilla JS
  for each(let value in hash) {
    process(value);
  } 

• Some SDK collections emulate object hashes that can be iterated via non-standard for but they don't fulfill object[key] expectations of standard JS.

  // Frankenstein JS
  for (var key in object) {
    // object[key] is not a value
  }
  
  // To get values you need Mozilla JS
  for each (let value in object) {
    process(value);
  }

Goals

• Use simplest (best to maintain) collection processing primitives.
• Eliminate alternative collection processing primitives in favor of conventional one.
• Minimize learning curve by using most common API used by target audience.
• Use same collection processing primitives for all types of collections.

Solution

There several possible solutions to address this:

(Mimic) Array methods

Use standard Array methods for array processing, as unlike alternatives they allow separation of concerns (mapping, filtering, reducing) and write wrappers for other collections implementing standard Array methods, preserving semantics of the wrapped data structures (iterators should remain lazy, enumerators should open on consumption):

var result = enumerable(enumerator).
  filter(predicate).
  map(f).
  reduce(accumulate);

var result = pairs(object).
  filter(predicate).
  map(f).
  reduce(accumulate);

This solution has some inherent constraints:

• No way to interrupt iteration.
• We will need wrapper per type, which adds learning curve.
• Use of methods is insecure, they can be tempered.

Standard underscore like API

Most popular JS frameworks jQuery on the web and [Underscore]most used on the server provide, standard Array like collection processing APIs. Also ES.next standard library will have @iter module with unbound Array-like methods for collection processing. This may be a perfect opportunity for us to solve mentioned problem in a common & well understood manner that most of our audience should be familiar with:

let { map, filter, reduce, chain } = require('sdk/collection')

var result = reduce(map(filter(enumerator, predicate), f), accumulate);

// Use underscore like `chain` for chaining
var value = chain(enumerator).
  filter(predicate).
  map(f).
  reduce(accumulate);

Note that this will also address some of the inherent problems of the first solution. For example underscore and jQuery both provide missing find operation that would stop iteration on first match. Also other less common primitives like take can be naturally layered when necessary. Security concerns are also solved by not relying on methods of instances.

Prior art

• underscore collections
• Traversing jQuery
• ES.next standard lib
• Clojure reducers

Discussion

https://etherpad.mozilla.org/jetpack-collections