Metricbeat SQL Helper

[sayden]

Some of our modules for SQL databases rely on plain SQL queries to fetch metrics information. Golang has the sql driver which should help in creating a SQL helper that just accepts queries and parses information for metricbeat.

There's a work already done in MS SQL metricbeat module that uses a similar approach, the first idea is to abstract this code from MSSQL module.

We have seen that if this is develop, the vendor folder in MySQL module should be removed https://github.com/elastic/beats/tree/master/metricbeat/module/mysql/vendor

We still need to discuss if this helper is going to be released only as helper of other modules or with a sql-module too but in any case, if a module if going to rely 90% of its functionality on this helper, it needs to follow a common modula GA checklist.

SQL Helper release checklist

This checklist is intended for Devs which create or update a module to make sure modules are consistent.

Modules

For a metricset to go GA, the following criterias should be met:

• Supported versions are documented
• Supported operating systems are documented (if applicable)
• Integration tests exist
• System tests exist
• Automated checks that all fields are documented
• Documentation
• Fields follow ECS and naming conventions
• Dashboards exists (if applicable)
• Kibana Home Tutorial (if applicable)

Metricbeat module

• Example data.json exists and an automated way to generate it exists (go test -data)
• Test environment in Docker exist for integration tests

For this particular Metricbeat Helper

• Remove vendor folder from MySQL module once it is migrated to use the helper

[elasticmachine]

Pinging @elastic/infrastructure

[sayden]

Here I'll leave some links to comments that will help in the development of this module / helper:

#9826 (comment)

[jsoriano]

#13257 has been merged with an initial version of a generic module for SQL and I have created #15048 as follow up. I am wondering now about the duplication with this issue.

Should we keep both and maintain this one for a helper? (a helper could still be helpful on the sql modules we already have)

Or we only continue only with the module and I close #15048 as duplicated of this issue (#9416) and get it updated.

[sayden]

Let's close this issue. It's older so it'll be less relevant. I think to better leave the Helper for a moment in future where we see a short-term need for it.