Ingest node (time) duration processor

[jakelandis]

Time duration's are often represented as Strings. Elasticsearch uses a simple way to define duration's. This is much simpler then the ISO durations.

This enhancement request is to create a duration processor that can read Elasticsearch's definition of duration's in String form and convert the that to a long type in the time period chosen. This processor would re-use Elasticsearch's time parsing and would be limited in scope to it's support. For example, since Elasticsearch does not currently support fractional duration's, neither would this processor.

An initial use case for this is for use of ingesting Elasticsearch's own logs which can log with duration's defined in this manner. For example, beats currently uses a custom painless script to handle this: https://github.com/elastic/beats/pull/9603/files#diff-0e8149f64b561ec89a4f9a01077b6f25R38. The introduction of this processor would allow for them to remove that custom code.

Future versions may be able to read ISO duration's too, but is out of scope for this request.

For example:

{
  "duration": {
    "field": "time.spent",
    "to_period": "seconds",
    "target_field": "time.duration_in_seconds"
  }
}

Could convert "1d" -> 86400

Related: #31737
Related: #31244

[elasticmachine]

Pinging @elastic/es-core-features

[dadoonet]

Could we also imagine something like:

{
  "duration": {
    "start_field": "begin",
    "end_field": "end",
    "to_period": "seconds",
    "target_field": "time.duration_in_seconds"
  }
}

So it will also help replacing custom painless script to compute a duration between 2 dates.

[jbaiera]

@dadoonet This is a pretty good idea, but considering the difference between durations (a change in time) and periods (a change in time tied to actual times), and in the interest of not over complicating a duration processor, I've opened #40615 for a processor that would handle that use case.

[lasiltan]

Am I correct in assuming this is entirely impossible in the current version (7.13)?

So I have a duration field with a value like P1DT1H4M and I would like to have a multi-field duration.millis with the duration parsed into a long.