[FileBeat] Fortinet module doesn't handle multiple IP addresses in dns.resolved_ip (ipaddr) field.

[fredtj]

Filebeat Fortinet module doesn't handle multiple IP addresses in dns.resolved_ip (ipaddr) field.

Sample log message:

timestamp=1591954984 tz="UTC+1:00" devname="xxx" devid="xxx" vd="root" date=2020-06-12 time=10:43:04 logid="1501054401" type="utm" subtype="dns" eventtype="dns-response" level="information" eventtime=1591954984675787304 tz="+0100" policyid=39 sessionid=38501592 user="xx" srcip=xx srcport=64848 srcintf="port2" srcintfrole="lan" dstip=8.8.8.8 dstport=53 dstintf="wan1" dstintfrole="wan" proto=17 profile="xx" xid=43289 qname="xx" qtype="A" qtypeval=1 qclass="IN" ipaddr="1.1.1.1, 1.1.1.2" msg="Domain was allowed because it is in the domain-filter list" action="pass" domainfilteridx=1 domainfilterlist="xxx"

note, field ipaddr is processed as follows:

    field: fortinet.firewall.ipaddr
    target_field: dns.resolved_ip
    ignore_missing: true

Error message:

Cannot index event publisher.Event
failed to parse field [dns.resolved_ip]
"reason":"'1.1.1.1, 1.1.1.2' is not an IP string literal."

The document here https://www.elastic.co/guide/en/ecs/current/ecs-dns.html says:

dns.resolved_ip | Array containing all IPs seen in answers.data. (snip) type: ip Note: this field should contain an array of values. example: ['10.10.10.10', '10.10.10.11']

I fixed this by adding a split processor to the utm pipeline as follows:

      {
        "split" : {
          "field" : "dns.resolved_ip",
          "separator" : ", ",
          "if" : "ctx.dns.resolved_ip.contains(',')"
        }
      },

I can submit PR if required?

[fredtj]

i hit another issue, probably Fortinet stupidness:

<190>timestamp=1590996323 date=2020-06-01 time=08:25:23 logver=2 type="traffic" level="info" sessionid=68723728 hostname="xxx" pcdomain=xxx uid="xxx" devname="xxx" devid="xxx" fgtserial="xxx" emsserial="xxx" regip=N/A srcname="svchost.exe" srcproduct="Microsoft® Windows® Operating System" srcip=[0000:fff:ffff:ffff:ffff:ffff:ffff:ffff] srcport=56916 direction="outbound" dstip=[ffff:ffff:ff:fff::ffff] remotename="N/A" dstport=443 user="SYSTEM@NT AUTHORITY" proto=6 rcvdbyte=N/A sentbyte=N/A utmaction="passthrough" utmevent="appfirewall" threat="OneDrive" vd="root" fctver="5.6.4.1131" os="Microsoft Windows 10 Professional Edition, 64-bit (build 18362)" usingpolicy="xxx" service="https" url="N/A" userinitiated=0 browsetime=N/A"

failed to parse field [destination.ip] of type [ip]
"reason":"'[ffff:ffff:ff:fff::ffff]' is not an IP string literal."

in this case it seems the fields srcip and dstip (both ipv6 only) are wrapped in square brackets?

[fredtj]

i fixed this one by the following changes to the traffic pipeline

      {
        "gsub": {
          "field": "destination.ip",
          "pattern": "\\[|\\]",
          "replacement": "",
          "if": "ctx.destination.ip.contains('[')"
        }
      },
      {
        "gsub": {
          "field": "source.ip",
          "pattern": "\\[|\\]",
          "replacement": "",
          "if": "ctx.source.ip.contains('[')"
        }
      },

I'm not sure if a) that is the optimal way to do it, and b) if these fixes need to go to the firewall pipeline, rather than traffic and utm? the latter could occur elsewhere although i've not seen it.

Regards

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[P1llus]

@fredtj Thanks for your feedback!

For your first question, adding that one in as we speak, together with part of the timestamp issue from earlier.

I have also created a separate issue for your second one, as that one seems to be forticlient, which I have not added as a fileset yet as I do not have any testdata for it.

#19314

For adding fortianalyzer support I have created this issue:

#19315