Filebeat merge several lines from mysql-slow.log into one line

[Vladpov]

Hello everyone,

I'm trying to analyze mysql-slow.log by using Filebeat Logstash and Elasticsearch. I have messages in mysql-slow.log file that look like this:

# Time: 2019-11-08T20:02:05.474508Z
# User@Host: user[user] @ localhost []  Id:     2
# Query_time: 0.000716  Lock_time: 0.000223 Rows_sent: 2  Rows_examined: 2
SET timestamp=1573243325;
select * from Persons;

First I try to make Filebeat send this log message with 5 lines to elasticsearch but all of them together in one line.

I set multiline input in filebeat.yml

multiline.pattern = `^\#`
multiline.negate = true
multiline.match = after

Unfortunately it doesn't work and elasticsearch recieves lines separately

1. message --> # Time: 2019-11-08T20:02:05.474508Z
2. message --> # User@Host: user[user] @ localhost [?] Id: 2 and so on...

I want to recieve it in one message in the following format:

# Time: 2019-11-08T20:02:05.474508Z # User@Host: user[user] @ localhost []  Id:     2 # Query_time: 0.000716  Lock_time: 0.000223 Rows_sent: 2  Rows_examined: 2 SET timestamp=1573243325; select * from Persons;

Thanks for any help!

[exekias (Carlos Pérez Aradros)]

Hi @Vladpov,

I'm wondering, did you try the MySQL module for this? It should take care of configuring the input for you, including slowlog: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-mysql.html

Best regards