Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metricbeat sql module: error mapping the query property #15736

Closed
narph opened this issue Jan 22, 2020 · 0 comments · Fixed by #15738
Closed

Metricbeat sql module: error mapping the query property #15736

narph opened this issue Jan 22, 2020 · 0 comments · Fixed by #15738
Assignees
Labels
Metricbeat Metricbeat Team:Integrations Label for the Integrations team v7.6.0

Comments

@narph
Copy link
Contributor

narph commented Jan 22, 2020

Initial ticket #13257

An example of event to be sent to es or other output looks like:

{ 
   "index":{ 
      "_index":"metricbeat-7.6.0"
   }
}{ 
   "@timestamp":"2020-01-22T10:20:53.035Z",
   "ecs":{ 
      "version":"1.4.0"
   },
   "host":{ 
      "name":"DESKTOP-RFOOE09",
      "hostname":"DESKTOP-RFOOE09",
      "architecture":"x86_64",
      "os":{ 
         "build":"18362.535",
         "platform":"windows",
         "version":"10.0",
         "family":"windows",
         "name":"Windows 10 Pro",
         "kernel":"10.0.18362.535 (WinBuild.160101.0800)"
      },
      "id":"1e50b6e1-9710-4164-a8f0-032b3c721dc3"
   },
   "agent":{ 
      "type":"metricbeat",
      "ephemeral_id":"b953f038-ca66-4fdc-99a8-44768ca8d753",
      "hostname":"DESKTOP-RFOOE09",
      "id":"4a1576f3-cd1f-4b8b-90d6-5918f5cb5d71",
      "version":"7.6.0"
   },
   "event":{ 
      "dataset":"sql.query",
      "module":"sql",
      "duration":76667500
   },
   "metricset":{ 
      "period":10000,
      "name":"query"
   },
   "service":{ 
      "address":"localhost:5433",
      "type":"sql"
   },
   "sql":{ 
      "driver":"postgres",
      "query":"select now()",
      "metrics":{ 
         "string":{ 
            "now":"2020-01-22 11:20:53.112"
         }
      }
   }
}

but in the fields.yml definition query is not a string property:

release: beta
  description: >
    query
  fields:
    - name: driver
      type: keyword
      description: >
        Driver used to execute the query.

ES is returning the errors:

","status":400,"error":{"type":"mapper_parsing_exception","reason":"object mapping for [sql.query] tried to parse field [query] as object, but found a concrete value"}}
@narph narph added Team:Integrations Label for the Integrations team Metricbeat Metricbeat v7.6.0 labels Jan 22, 2020
jsoriano added a commit to jsoriano/beats that referenced this issue Jan 22, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier check documented fields.

Fixes elastic#15736
jsoriano added a commit that referenced this issue Jan 22, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier detect this kind of issues.

Also add validation so driver and sql_query options cannot be empty.

Fetch was reusing some maps when building the event, what was
causing mixed data in events and panics on queries with lots of rows.

Fixes #15736
jsoriano added a commit to jsoriano/beats that referenced this issue Jan 22, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier detect this kind of issues.

Also add validation so driver and sql_query options cannot be empty.

Fetch was reusing some maps when building the event, what was
causing mixed data in events and panics on queries with lots of rows.

Fixes elastic#15736

(cherry picked from commit badb2cb)
jsoriano added a commit to jsoriano/beats that referenced this issue Jan 22, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier detect this kind of issues.

Also add validation so driver and sql_query options cannot be empty.

Fetch was reusing some maps when building the event, what was
causing mixed data in events and panics on queries with lots of rows.

Fixes elastic#15736

(cherry picked from commit badb2cb)
jsoriano added a commit that referenced this issue Jan 22, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier detect this kind of issues.

Also add validation so driver and sql_query options cannot be empty.

Fetch was reusing some maps when building the event, what was
causing mixed data in events and panics on queries with lots of rows.

Fixes #15736

(cherry picked from commit badb2cb)
jsoriano added a commit that referenced this issue Jan 23, 2020
Mapping was defined at the metricset level, but it should be defined at
the module level.

Add system test to earlier detect this kind of issues.

Also add validation so driver and sql_query options cannot be empty.

Fetch was reusing some maps when building the event, what was
causing mixed data in events and panics on queries with lots of rows.

Fixes #15736

(cherry picked from commit badb2cb)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metricbeat Metricbeat Team:Integrations Label for the Integrations team v7.6.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants