Network direction processor

[andrewstucki]

This PR computes the ECS perimeter-based network direction of a document given a source and destination ip and a list of ip addresses that specify the internal network boundary. It's essentially a port of the beats processor along with named range support.

➜  ~ curl -H "Content-Type: application/json" -X POST -u elastic:password http://localhost:9200/_ingest/pipeline/_simulate\?verbose --data-binary @- << EOF
{
  "pipeline": {
    "processors": [
      {
        "network_direction": {
          "internal_networks": [ "private" ]
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "source": {
          "ip": "128.232.110.120"
        },
        "destination": {
          "ip": "192.168.1.1"
        }
      }
    }
  ]
}
EOF
{"docs":[{"processor_results":[{"processor_type":"network_direction","status":"success","doc":{"_index":"_index","_id":"_id","_source":{"destination":{"ip":"192.168.1.1"},"source":{"ip":"128.232.110.120"},"network":{"direction":"inbound"}},"_ingest":{"pipeline":"_simulate_pipeline","timestamp":"2020-12-18T20:23:46.589327Z"}}}]}]}%

See "network":{"direction":"inbound"}

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[danhermann]

@andrewstucki, thanks for submitting this. I'll review it from the ES Core Features side, but it might wait until early January since our team has limited bandwidth until then.

[andrewstucki]

@danhermann that's perfectly fine, I'm hoping it can get in by 7.12 so that it can unblock some ECS upgrade work we postponed on the beats ingest side of things, but no rush. Just whipped this up to gain a bit more familiarity with ES.

[danhermann]

@elasticmachine update branch

[danhermann]

@andrewstucki, there are some checkstyle violations here that need to be addressed. You should be able to see them either in the output of the elasticsearch-ci/2 build or by running ./gradlew :x-pack:plugin:ingest:checkstyleTest locally.

[andrewstucki]

@danhermann sorry about the late response. I merged latest master in so this wasn't stale anymore. The checkstyle issues seemed to be outdated and related to a .* import that I initially had. For some reason I don't think CI picked up the change when I fixed it. Hopefully we'll see the build pass this time around.

[danhermann]

@andrewstucki, this looks good. I added two very minor comments below and once those are addressed, I can get this merged.

[danhermann]

Thanks, @andrewstucki. This looks good now and I'll get it merged into the appropriate branches.

[danhermann]

cc: @elastic/es-ui in case Kibana auto-complete needs to be updated with this new processor.