[filebeat/postgresql] Multiline pattern is too permissive/generic

[tsouza]

The multiline pattern used for postgres ^[-0-9]* is too permissive and it will match a line that starts with a space.

The following is a sample postgres log that fails creating proper multiline event since it has log lines that starts with a space:

2019-05-06 19:00:04.511 UTC [913763] elastic@opbeans LOG:  duration: 0.753 ms  statement: SELECT p.id, p.sku, p.name, p.stock, t.name AS type_name FROM products p LEFT JOIN product_types t ON p.type_id=t.id
    FROM orders JOIN customers ON orders.customer_id=customers.id
    FROM products JOIN product_types ON type_id=product_types.id

I think the test data here https://github.com/elastic/beats/blob/v7.0.0/filebeat/module/postgresql/log/test/postgresql-9.6-debian-with-slowlog.log does not catches this error because it starts with a tab instead of a space char, so the sample there fails ^[-0-9]* .

The recommended fix here is to use a more specific date pattern like ^\d{4}-\d{2}-\d{2} . Alternatively, use + instead of *, like in ^[-0-9]+ , which does not allows lines starting with a space.

Filebeat Version: 7.0.0
Postgres Version: 9.6.2

[sayden]

Hi @tsouza and thank you for the heads up 🙂

I have been experimenting with your suggestions and I have finally opened a PR to solve this here #13069

Feel free to comment and thank you again 🙂

[sayden]

Closing this after PR's were merged