Fix for #1382 - logs are not easily queryable with es storage

[sobvan]

As logs was not defined as a nested object. Nested elastic query did not work for logs.fields either. This commit fixes this issue:
resolves #1382

Which problem is this PR solving?

#1382 - Logs are not easily queryable from elasticsearch (ES6)

Short description of the changes

Added two lines to elastic jaeger span index definition - see in changes.

Note

I do not know how to create unit tests for this. If unit tests are needed for this PR and there is someone who can guide me a bit, I would really appreciate that.

[yurishkuro]

Jaeger backend can query for them, what's different there?

@pavolloffay is this a breaking change?

[sobvan]

Maybe it is ES 6 related. Without the nested type for logs filed, the nested query did not work. I assume jaeger backend just uses the json returned for each span document, and renders it on the UI. I am not aware of any log browser other than the trace view.
I am about to implement a log simple browser in one of our company apps, because I really like the idea of span context based logging.
It might also be related to #649.

[pavolloffay]

@istvanszoboszlai We have to make sure this works with ES5 and ES6.

Do not merge I will have a more close look at this.

[pavolloffay]

I have tested the current master (without this PR) with the following query and everything works with ES 5.x and 6.x, therefore I am not sure if we need this change.

The data has been generated with hotrod example.

curl -H "Content-Type: application/json" localhost:9200/jaeger-span-*/_search?filter_path=took,hits.hits._source  -d '{                                                 12:44 
  "_source": ["logs.timestamp", "logs.fields.key", "logs.fields.tagType", "logs.fields.value"],
  "query": {
    "nested": {
      "path": "logs.fields",
      "query": {
        "bool": {
          "must": [
            {"wildcard": { "logs.fields.value": "GetConn" }}
            ]
        }
      }
    }
  }
}'

Example span

{
        "_index" : "jaeger-span-2019-06-21",
        "_type" : "span",
        "_id" : "AWt5lQrDcFs-LnUWHqwX",
        "_score" : 1.0,
        "_source" : {
          "traceID" : "7e14ff6b00ea658b",
          "spanID" : "59f53c718b65410f",
          "flags" : 1,
          "operationName" : "HTTP GET",
          "references" : [
            {
              "refType" : "CHILD_OF",
              "traceID" : "7e14ff6b00ea658b",
              "spanID" : "603c8eb2c176f08"
            }
          ],
          "startTime" : 1561112938308616,
          "startTimeMillis" : 1561112938308,
          "duration" : 69196,
          "tags" : [
            {
              "key" : "span.kind",
              "type" : "string",
              "value" : "client"
            },
            {
              "key" : "component",
              "type" : "string",
              "value" : "net/http"
            },
            {
              "key" : "http.method",
              "type" : "string",
              "value" : "GET"
            },
            {
              "key" : "http.url",
              "type" : "string",
              "value" : "http://0.0.0.0:8083/route?dropoff=577%2C322&pickup=936%2C463"
            },
            {
              "key" : "http.url",
              "type" : "string",
              "value" : "0.0.0.0:8083"
            },
            {
              "key" : "net/http.reused",
              "type" : "bool",
              "value" : "true"
            },
            {
              "key" : "net/http.was_idle",
              "type" : "bool",
              "value" : "true"
            },
            {
              "key" : "http.status_code",
              "type" : "int64",
              "value" : "200"
            },
            {
              "key" : "internal.span.format",
              "type" : "string",
              "value" : "proto"
            }
          ],
          "logs" : [
            {
              "timestamp" : 1561112938308666,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GetConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938308674,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GotConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938308721,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "WroteHeaders"
                }
              ]
            },
            {
              "timestamp" : 1561112938308724,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "WroteRequest"
                }
              ]
            },
            {
              "timestamp" : 1561112938377614,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GotFirstResponseByte"
                }
              ]
            },
            {
              "timestamp" : 1561112938377752,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "PutIdleConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938377812,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "ClosedBody"
                }
              ]
            }
          ],
          "process" : {
            "serviceName" : "frontend",
            "tags" : [
              {
                "key" : "jaeger.version",
                "type" : "string",
                "value" : "Go-2.15.1dev"
              },
              {
                "key" : "hostname",
                "type" : "string",
                "value" : "dhcp-144-223.gva.redhat.com"
              },
              {
                "key" : "ip",
                "type" : "string",
                "value" : "10.33.144.223"
              },
              {
                "key" : "client-uuid",
                "type" : "string",
                "value" : "6bcf23acd0fff989"
              }
            ]
          }
        }
      },

The output of query from ES:
out.txt

[pavolloffay]

Although I am fine to add the type: "nested" as it is really a nested type.

https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html

The nested type is a specialised version of the object datatype that allows arrays of objects to be indexed in a way that they can be queried independently of each other.

"logs" : [
            {
              "timestamp" : 1561112938308666,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GetConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938308674,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GotConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938308721,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "WroteHeaders"
                }
              ]
            },
            {
              "timestamp" : 1561112938308724,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "WroteRequest"
                }
              ]
            },
            {
              "timestamp" : 1561112938377614,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "GotFirstResponseByte"
                }
              ]
            },
            {
              "timestamp" : 1561112938377752,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "PutIdleConn"
                }
              ]
            },
            {
              "timestamp" : 1561112938377812,
              "fields" : [
                {
                  "key" : "event",
                  "type" : "string",
                  "value" : "ClosedBody"
                }
              ]
            }
          ],

[pavolloffay]

I have submitted #1622 as this PR needs to be signed and we want to release soon.

[pavolloffay]

Done in #1622.

@istvanszoboszlai thanks for reporting the issue and submitting the PR.

[sobvan]

Done in #1622.

@istvanszoboszlai thanks for reporting the issue and submitting the PR.

Thank you @pavolloffay for taking care of this.