Allow expressions as elements in array output values

[samanpwbb]

mapbox-gl-js version: v0.44.0

It would be great if I could use expressions in each element of an array value for properties like text-font and *-translate.

Lets take text-offset for example. The output value is a 2-element array. As a literal, can be written like this:

[0,1]

Lets imagine a hypothetical situation where I wanted to offset along the y axis based on the existence of a property in my data. This would be a useful feature if I had icons for some labels but not for others. I can write this expression like so, and it's valid:

[
  "case",
  ["has", "icon"],
  ["literal", [0, -10]],
  ["literal", [0, 0]]
]

If I have more than one case where I want to adjust my y axis value (say, to check for the existence of 'big-icon' and 'small-icon'), the syntax gets even more unwieldy:

[
  "case",
  ["has", "icon-small"],
  ["literal", [0, -10]],
  ["has", "icon-large"],
  ["literal", [0, -20]],
  ["literal", [0, 0]]
]

It would be amazing if the following was valid:

[
  0,
  [ "case", 
    ["has", "icon-small"],
    -10
    ["has", "icon-large"],
    -20,
    0
  ]
]

Now, that probably won't work because of the way the expression syntax is designed. Could a to-array expression get me what I want?

[
  "to-array",
  0,
  [ "case", ["has", "icon"], -10, 0]
]

This would be a really valuable feature for many properties, and particularly text-font, where usually users will have a universal fallback font, but would want to use an expression to dictate what the primary font is. It would also lead to a more user-friendly experience in Studio!

[anandthakker]

This is very doable implementation-wise, once we settle on the right API design.

What should we call it? to-array is inconsistent, because the other to-* expressions coerce a single value to the given type. ["array-from", 1, 2, 3]? ["make-array", 1, 2, 3]? I don't like either of these...

As a variation on the proposal above, we could have an analogue to literal that evaluates each array item / object value -- e.g. [ "semi-literal", [ ["get", "x-offset"], ["get", "y-offset"] ] ] to produce a value like [ xoffset, yoffset ] or [ "semi-literal", { a: ["get", "blah"], b: ["get", "blah2"] }] to produce a value like { a: .., b: ..}. (But again, no idea what to name this...)

[samanpwbb]

I think the second style, ["operator", [...]] makes sense! With that syntax in mind, could we call it to-array? So, ["to-array", [1,2,3]] = [1,2,3] ?

[jfirebaugh]

The lisp tradition (which has quite a bit of smart thinking behind it) is to have two special forms: quote (same as our literal) and quasiquote (similar but slightly different from @anandthakker's semi-literal proposal). For example: http://www.gnu.org/software/mit-scheme/documentation/mit-scheme-ref/Quoting.html

[stevage]

Couple of comments.

First, this form doesn't seem unwieldy to me:

[
  "case",
  ["has", "icon-small"], ["literal", [0, -10]],
  ["has", "icon-large"], ["literal", [0, -20]],
  ["literal", [0, 0]]
]

It has the advantage that the [0, -10] stop syntax is clearly visible.

If, perchance, the requirement for "literal" were dropped and arrays starting with numbers were automatically treated as either numbers or numeric arrays, this gets even better:

[
  "case",
  ["has", "icon-small"], [0, -10],
  ["has", "icon-large"], [0, -20],
  [0, 0]
]

By the same logic, your "it would be amazing if this worked" should work:

[
  0,
  [ "case", 
    ["has", "icon-small"], -10
    ["has", "icon-large"], -20,
    0
  ]
]

I guess I don't really understand the constraints that make this impossible. Is there any public writing about the new expression syntax that explains why there has to be these extra layers of assertions ("number", "array"), conversions ("to-number", "to-array"), getter functions ("get") etc? (I really, really want to be on board with expressions but haven't really got there yet :/ )

[anandthakker]

If, perchance, the requirement for "literal" were dropped and arrays starting with numbers were automatically treated as either numbers or numeric arrays
I guess I don't really understand the constraints that make this impossible.

@stevage This isn't impossible -- nor is, say, accepting object literals without wrapping them with ["literal", ...] -- but, with the expression syntax being relatively new, we want to be somewhat conservative about the syntax so that we're not overly constrained when we want to make revisions to it in the future. (Personally, I also think that it's just much easier to communicate the rule "all array literals have to be wrapped in [literal,...]," than the rule "all array literals that start with a string have to be wrapped in [literal, ...]` -- but that's definitely debatable, and not the main reason anyway.)

@jfirebaugh yeah, I thought about quasiquote too, but I'm not sure it's warranted here -- I think it's probably sufficient to just have an equivalent for the more basic list function that doesn't require unquoting the items that should be evaluated. (list (+ 1 2) (+ 3 4)). AFAICT, quasiquote would be most useful for making arrays with many items being nested array literals and just a few items being expressions that should be evaluated -- that seems like a use case that's much more common in Lisp than in our context.

[brncsk]

I bumped into this today while trying to compute text-offset dynamically for placing pie chart labels (I know...). Pre-computing these at tile generation time would be an option, but I'm not sure if this use case (namely, array-valued properties in vector tiles) is supported. (If it isn't, which I assume to be the case – does that also mean that data-driven rendering of these style properties is only possible using GeoJSON sources?).

[samanpwbb]

If, perchance, the requirement for "literal" were dropped and arrays starting with numbers were automatically treated as either numbers or numeric arrays, this gets even better:

This would be the absolute best solution from my perspective. It would allow us to use all our specialized array widgets in Studio directly inside nested expressions. Fair if there are reasons why this is not feasible, but it'd make the Studio team's job a lot easier.

[anandthakker]

Note that even with the proposal for accepting arrays starting with a non-string without the literal wrapper, you’d still have to use ["literal", [...]] if you wanted an array of strings.

[samanpwbb]

Small update here: We wrote a workaround in Studio so we can support nice UI widgets to edit the output value of a simple literal expression, so no urgency from our end on addressing this issue.

[jstratman]

The ability to use arrays directly from geojson sources has been great via "text-offset": ["get","offset"], but assuming that'll never be feasible with vector tiles, it seems this proposal would open up a similar level of flexibility for expressions broadly.

@anandthakker any plans to move forward with this approach?

A style I have in mind currently has 190+ stops for each offset combination and it gets challenging to maintain, though I'm unsure if it's negatively impacting performance at this point.

Being able to just do this, or other embedded expressions, seems intuitive enough to me:

[
  ["get", "x-offset"],
  ["get", "y-offset"]
] 

[anandthakker]

@jstratman we should, indeed, move forward on this, but we need to decide on the API design. Main two proposals on the table:

1. ["semi-literal", [["get", "x"], ["get", "y"]] / ["semi-literal", { "x": ["get", "x"], "y": ["get", "y"] }] Allow expressions as elements in array output values #6155 (comment)
2. Just evaluate any array that doesn't start with a string: [["get", "x"], ["get", "y"]] Allow expressions as elements in array output values #6155 (comment). If you wanted an array starting with a string literal (e.g. ["apple", ["get", "banana"], "pear"], you'd have to escape it somehow, since "apple" otherwise be interpreted as an operator. Maybe something like: [["concat", "apple"], ["get", "banana"], "pear"]. Or maybe using double brackets? [[ "apple", ["get, "banana"], "pear" ]]

@samanpwbb @jfirebaugh thoughts?

[anandthakker]

A third way would be to just always use [[ ]]. I kinda like that the best, actually.

[1ec5]

It would also be great if objects could contain expressions. For example, the collator expression’s first argument is an object rather than an array.

Historically, the iOS and macOS SDKs represented function stops as dictionaries (akin to JSON objects), so it’s natural for developers migrating from style functions to attempt to index into an object. Even without migrating from style functions, dictionaries are the natural way to build a variable-length match expression. This works in general but not for types like colors that can’t be represented literally in JSON without an expression. mapbox/mapbox-gl-native#11830 has a representative example of this use case, as well as a workaround.

[jfirebaugh]

A third way would be to just always use [[ ]].

That could work, although I'm wary of introducing another form of syntax. Another option would be to go back to the basic array-from/make-array idea but spell it []:

["[]", ["get", "x"], ["get", "y"]]

It could be accompanied by a corresponding {} which requires an even number of arguments.

["{}",
    "key_x", ["get", "x"],
    "key_y", ["get", "y"]]

[anandthakker]

dictionaries are the natural way to build a variable-length match expression

@1ec5 the "match" expression in the style spec is specifically designed for this case. Ideally, I think we'd want "match" (the NSExpression representation of it) to be the most natural way to build a matching expression.

[1ec5]

the "match" expression in the style spec is specifically designed for this case. Ideally, I think we'd want "match" (the NSExpression representation of it) to be the most natural way to build a matching expression.

Sort of. No matter how we design expressions, the fact is that dictionary lookups are how Objective-C and Swift developers approach the problem, as opposed to an array of alternating keys and values. So what I’ve shared in mapbox/mapbox-gl-native#11830 (comment) is a way to build a match expression from a dictionary.

[jhwegener]

Any news on this issue?

[Argh4k]

Hi, any updates?

[itbeyond]

Any update on this?

[taetscher]

Hi, is this still in the workings?

[arthureffting]

I feel like this should have been included by now. Anyone working on this?

[the-nemz]

Hello, following up again here. It would be fantastic to use data-driven properties to set things like *-translate. Has any progress been made yet? @anandthakker

[jayarjo]

Ping!

[Racquetballer]

Same here. I have offset, offsetX, offsetY values in properties that I need to use for icon-offset values. Since tilesets seem to not allow storing arrays in properties they are converted to strings. "[0,100]". How can I set icon-offset using expressions based on my offsetX and offsetY numbers?

This doesn't seem to work.

map.setLayoutProperty('pc-campground-attributes', 'icon-offset', ["literal", [["get", "offsetX"],["get", "offsetY"]]]);

[RobinSchwaller]

Ping!

[quantenschaum]

side question: Is it possible to parse a JSON string to an array somehow?

[stevage]

Just ran into this again - really wanted to be able to dynamically calculate a value for text-offset.

It seems there were half a dozen reasonable options on the table, and nothing got implemented because none stood out as the best?