Mysql-slow multiline

Elastic Stack Logstash
1

Hello,

I am parsing the mysql-slow logs using the parser:

input{
	pipeline {
		address => input
	}
                file{
                path => "/etc/logstash/logsamples/all.log"
          start_position => "beginning"
          sincedb_path => "/dev/null"
	  codec => multiline {
          pattern => "^ # User @ Host:"
          negate => true
          what => "previous"
   }
       }
}

filter{
	grok{
	match => {"message" =>"%{SYSLOGBASE2}%{GREEDYDATA:mess}"}
	pattern_definitions => {
	"GREEDYMULTILINE" => "(\r|\n)*"
}
	}
}

As result i get all the logs bellow as one line and not multiline.

Jun 1 01:01:58 mypc mysql-slow: # Time: 210601 1:01:58
Jun 1 01:01:58 mypc mysql-slow: # User@Host: fp[fp] @ [10.64.5.169]
Jun 1 01:01:58 mypc mysql-slow: # Thread_id: 91926893 Schema: oli QC_hit: No
Jun 1 01:01:58 mypc mysql-slow: # Query_time: 2.769906 Lock_time: 0.000126 Rows_sent: 0 Rows_examined: 14893
Jun 1 01:01:58 mypc mysql-slow: # Rows_affected: 0
Jun 1 01:01:58 mypc mysql-slow: SET timestamp=1622502118;
Jun 1 01:01:58 mypc mysql-slow: SELECT id, amount
Jun 1 01:01:58 mypc mysql-slow: FROM dbo
Jun 1 01:01:58 mypc mysql-slow: WHERE
Jun 1 01:01:58 mypc mysql-slow: idOperationType = '4'
Jun 1 01:01:58 mypc mysql-slow: AND idAccount = '228'
Jun 1 01:01:58 mypc mysql-slow: AND receiveDate >= '2021-05-31 00:00:00'
Jun 1 01:01:58 mypc mysql-slow: AND receiveDate < '2021-06-01 00:00:00'
Jun 1 01:01:58 mypc mysql-slow: AND bk IN ('9','2');
Jun 1 01:02:13 mypc mysql-slow: # Time: 210601 1:02:12
Jun 1 01:02:13 mypc mysql-slow: # User@Host: fp[fp] @ [10.64.5.169]
Jun 1 01:02:13 mypc mysql-slow: # Thread_id: 91926889 Schema: lifa3 QC_hit: No
Jun 1 01:02:13 mypc mysql-slow: # Query_time: 2.898778 Lock_time: 0.000064 Rows_sent: 0 Rows_examined: 21134
Jun 1 01:02:13 mypc mysql-slow: # Rows_affected: 0
Jun 1 01:02:13 mypc mysql-slow: use lifa3;
Jun 1 01:02:13 mypc mysql-slow: SET timestamp=1622502132;
Jun 1 01:02:13 mypc mysql-slow: SELECT id, amount
Jun 1 01:02:13 mypc mysql-slow: FROM dbo
Jun 1 01:02:13 mypc mysql-slow: WHERE
Jun 1 01:02:13 mypc mysql-slow: idOperationType = '4'
Jun 1 01:02:13 mypc mysql-slow: AND idAccount = '321'
Jun 1 01:02:13 mypc mysql-slow: AND receiveDate >= '2021-05-31 00:00:00'
Jun 1 01:02:13 mypc mysql-slow: AND receiveDate < '2021-06-01 00:00:00'
Jun 1 01:02:13 mypc mysql-slow: AND bk IN ('9','2');

can you please help me to solve the issue?

Best regards

2

Hello expert,

do you have any proposals to solve the issue.

Best regards,

3

I am surprised you get anything at all. That pattern does not appear in your logs, so it should read the entire file as one event which never gets flushed, because it only gets flushed when that pattern matches.

If you log entries look like

Jun 1 01:02:13 mypc mysql-slow: # User@Host: fp[fp] @ [10.64.5.169]

then change the pattern to pattern => "# User@Host: " (unanchored). You may also want to add auto_flush_interval => 2 to the codec.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.