add mariadb module for filebeat

[rdglinux]

Added filebeat module to mariadb logs.

Module created based on discuss: https://discuss.elastic.co/t/filebeat-mysql-module-slowlog-error-message-provided-grok-expressions-do-not-match-field-value/135945/3

Logging pattern to be parsed:

# Time: 180613 11:04:36
# User@Host: root[root] @ localhost [ ]
# Thread_id: 5  Schema:   QC_hit: No
# Query_time: 2.000652  Lock_time: 0.000000  Rows_sent: 1  Rows_examined: 0
SET timestamp=1528898676;
select sleep(2);

To do:

Adjust and create dashboard file do Kibana

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[karmi]

Hi @rdglinux, we have found your signature in our records, but it seems like you have signed with a different e-mail than the one used in yout Git commit. Can you please add both of these e-mails into your Github profile (they can be hidden), so we can match your e-mails to your Github profile?

[jsoriano]

@rdglinux thanks for this! You will need to run make update inside filebeat and commit the changes.

[ruflin]

Could you also add a small test log file to verify the changes (see other filesets). PR will also need a changelog entry.

[rdglinux]

@karmi thanks for your reply...i updated my other e-mail in my github.

[karmi]

@rdglinux , great!, the CLA check is green now.

[ruflin]

@rdglinux Something seems to be still off with the generated files. Could you rebase on master and run again make update ?

[ceesvanegmond]

Cool new feature! Will it get merged?

[ceesvanegmond]

Fix works on my machine!

[jsoriano]

@rdglinux are you planning to continue with this PR? if you cannot do it we could try to finish and merge it. It'd be great in any case if you could add some example log file for testing. Thanks!

[rdglinux]

Hi @jsoriano!

Sorry for my late reply, i was involved in other projects.

I used this module for my database MariaDB and works good. So i belive we could merge it.

Fallow my log!.

[NoSkillGuy]

Hi @rdglinux

My MariaDB slow log format is slightly different.

# Time: 181004 10:53:53
# User@Host: user_name[current_user_name] @  [ip]
# Thread_id: 25844  Schema: blah  QC_hit: No
# Query_time: 178.306017  Lock_time: 0.000000  Rows_sent: 0  Rows_examined: 53022772
# Rows_affected: 3062
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: 0  Priority_queue: No
SET timestamp=1538650433;
call PROC('blah');

Version Details:
mysql Ver 15.1 Distrib 10.2.12-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

[NoSkillGuy]

Hi @rdglinux, For the above format I had written a new grok pattern

"^#%{SPACE}Time:%{SPACE}%{NUMBER:microseconds}%{SPACE}%{NUMBER:hours}:%{NUMBER:minutes}:%{NUMBER:seconds}\n#%{SPACE}User@Host:%{SPACE}%{USER:mysql.slowlog.user}\\[%{USER:current_user}\\]%{SPACE}@%{SPACE}\\[%{IP:mysql.slowlog.ip}\\]\n#%{SPACE}Thread_id:%{SPACE}%{NUMBER:thread_id}%{SPACE}Schema:%{SPACE}%{WORD:schema}%{SPACE}QC_hit:%{SPACE}%{WORD:qc_hit}\n#%{SPACE}Query_time:%{SPACE}%{NUMBER:mysql.slowlog.query_time.sec}%{SPACE}Lock_time:%{SPACE}%{NUMBER:mysql.slowlog.lock_time.sec}%{SPACE}Rows_sent:%{SPACE}%{NUMBER:mysql.slowlog.rows_sent}%{SPACE}Rows_examined:%{SPACE}%{NUMBER:mysql.slowlog.rows_examined}\n#%{SPACE}Rows_affected:%{SPACE}%{NUMBER:rows_affected}\n#%{SPACE}Full_scan:%{SPACE}%{WORD:full_scan}%{SPACE}Full_join:%{SPACE}%{WORD:full_join}%{SPACE}Tmp_table:%{SPACE}%{WORD:tmp_table}%{SPACE}Tmp_table_on_disk:%{SPACE}%{WORD:tmp_table_on_disk}\n#%{SPACE}Filesort:%{SPACE}%{WORD:filesort}%{SPACE}Filesort_on_disk:%{SPACE}%{WORD:filesort_on_disk}%{SPACE}Merge_passes:%{SPACE}%{NUMBER:merge_passes}%{SPACE}Priority_queue:%{SPACE}%{WORD:priority_queue}\nSET%{SPACE}timestamp=%{NUMBER:mysql.slowlog.timestamp};\n%{GREEDYMULTILINE:mysql.slowlog.query}"

[jsoriano]

@rdglinux sorry also for my late reply 🙁

This is looking good to me, could you try to add some example test files? It'd be good if this works also with @NoSkillGuy's example.

[SPR0STO]

Hi @rdglinux and @jsoriano
When can we expect new functionality in production?

[jsoriano]

MariaDB support added at the end in the mysql module itself (#9731), thanks for the work and example cases here!
I will try to backport this for 6.7.0.