2020-08-04-3325-remap-v2.md

RFC #3325 - Remap V2

Scope

This RFC covers incremental changes to the remap mapping syntax. Previous RFC for context: https://github.com/vectordotdev/vector/blob/master/rfcs/2020-07-21-2744-remap-syntax.md

Motivation

We have a remap transform that allows basic mutation operations:

.foo = .bar
del(.bar)
.baz = "something static"

However, large mappings almost always require conditional logic and implementing this with the swimlanes transform introduces complexity to a config. Ideally, we want to be able to express conditional mappings directly within the remap transform using an if statement.

Doc-level Proposal

Within a remap mapping it's possible to perform mappings conditionally using an if statement:

if .bar != null {
  .foo = .bar
  del(.bar)
}
.baz = "something static"

When the if logical argument resolves to true the mapping statements within the block are executed, otherwise they are not. An else block can be added, where the contents are executed when the argument resolves to false:

if .bar != null {
  .foo = .bar
  del(.bar)
} else {
  .foo = "bar doesn't exist"
}
.baz = "something static"

Sometimes it might be more convenient to express if statements as an expression for assigning to a value:

.content = if .type == "foo" { .foo.content } else { .bar.body }

In which case each block must contain a query. It would be possible to expand this to allow assignments within the block but we need to think through and experiment with the issues of mutating context and how we'd document the behavior we choose.

Plan of Attack

1. Implement root-level if statements, this is the lowest hanging fruit in terms of implementation and documentation and unblocks the majority of use cases.
2. Implement value if statements which can exist on the right-hand side of an assignment.
3. Consider allowing assignments within right-hand side if statement blocks. This could potentially become an RFC of its own.