I am trying to collect MariaDB slow query logs with Filebeat and group each query as a single event. My logs look like this:
SET timestamp=1764062324;
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'HORIZONEU_archive'
AND TABLE_NAME = 'KIT_ASSEMBLIES';
# User@Host: dba[dba] @ [10.71.180.40]
# Thread_id: 213322 Schema: admin QC_hit: No
...
SET timestamp=1764062324;
SELECT TRIM(ROUTINE_COMMENT) AS PROCEDURE_HASH
FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_SCHEMA = 'HORIZONEU_stage'
AND ROUTINE_NAME = 'sp_LOAD_KIT_ASSEMBLIES_CDC';
# User@Host: dba[dba] @ [10.71.180.40]
...
I want each SET timestamp=... block (plus all lines until the next SET timestamp=) to be sent as a single event to Graylog.
filebeat.inputs:
- type: filestream
enabled: true
paths:
- /var/log/mariadb/slow.log
fields:
type: mariadb-slow-log
source_type: mariadb_slow
multiline:
pattern: '^SET timestamp='
match: after
negate: true
output.logstash:
hosts: [""]
I also tried:
multiline:
pattern: '^SET timestamp=|^# User@Host:'
match: after
negate: true
What I observe:
-
Filebeat still sends each line as a separate event.
-
Graylog receives one JSON per line instead of grouped blocks.
Request:
-
Advice on correct multiline pattern for MariaDB slow query logs.
-
Explanation why
pattern: '^SET timestamp='or'^SET timestamp=|^# User@Host:'does not work.