I have the following problem:
This is my logfile
11:30:00,909 9=DEB BswTcpTskComSrv::passTele()
11:30:00,909 12=EVT | TELEGRAMM SEND comsrv->TskEvm: |TskBrm|TskEvm|TeleDb|
11:30:00,909 12=EVT | BswTcpTskComSrv::sendTeleTo:23 Bytes gesendet
11:30:00,909
11:30:00,909 LOAD STATISTICS
11:30:00,909
11:30:00,909 =========================+================================+============+========================================+=====================
11:30:00,909 | % BUSY | TOTAL | TELEGRAMS |
11:30:00,909 TASK | MIN 10MIN HOUR TOTAL | TIME [s] | MIN 10MIN HOUR TOTAL OPEN | UP SINCE
11:30:00,909 -------------------------+--------------------------------+------------+----------------------------------------+---------------------
11:30:00,909 DEEFR0004 | 0.20 0.18 0.18 0.23 | 0.433 | 2 6 6 6 0 | 29.10.2020 11:26:52
11:30:00,909 DEEFR0046 | 0.00 0.00 0.02 0.28 | 44.440 | 0 0 10 550 0 | 29.10.2020 07:01:54
11:30:00,909 DEEFR0051 | 0.76 0.98 1.21 1.27 | 54.578 | 2 64 294 436 0 | 29.10.2020 10:18:37
11:30:00,910 DEEFR0082 | 0.00 3.70 2.80 1.68 | 356.703 | 0 294 1304 3988 0 | 29.10.2020 05:36:39
11:30:00,910 DEEFR0083 | 0.00 0.00 0.00 0.10 | 20.151 | 0 0 0 280 0 | 29.10.2020 05:37:32
11:30:00,910 DEEFR0086 | 0.00 0.00 0.00 0.21 | 33.894 | 0 0 0 350 0 | 29.10.2020 06:59:41
11:30:00,910 DEEFR0093 | 0.00 0.00 0.00 0.24 | 39.455 | 0 0 0 480 0 | 29.10.2020 07:00:33
11:30:00,910 DEEFR0102 | 0.00 0.00 0.00 0.45 | 94.647 | 0 0 0 1166 0 | 29.10.2020 05:36:30
11:30:00,910 DEEFR0107 | 0.00 0.22 0.15 0.23 | 39.704 | 0 16 84 506 0 | 29.10.2020 06:40:16
11:30:00,910 DEEFR0114 | 0.00 0.09 0.66 0.64 | 73.202 | 0 8 288 854 0 | 29.10.2020 08:18:58
11:30:00,910 DEEFR0130 | 0.00 0.00 0.00 0.07 | 11.304 | 0 0 0 180 0 | 29.10.2020 07:04:53
11:30:00,910 DEEFR0141 | 0.00 0.00 0.00 0.00 | 0.292 | 0 0 0 2 0 | 29.10.2020 08:30:10
11:30:00,910 DEEFR0147 | 0.00 1.13 0.71 0.91 | 55.629 | 0 90 294 646 0 | 29.10.2020 09:48:26
11:30:00,910 DEEFR0164 | 0.00 0.00 0.00 0.13 | 15.560 | 0 0 0 182 0 | 29.10.2020 08:03:41
11:30:00,910 DEEFR0165 | 0.00 0.64 0.17 0.11 | 23.701 | 0 18 40 314 0 | 29.10.2020 05:35:51
11:30:00,910 DEEFR0168 | 0.70 0.12 0.69 0.63 | 46.330 | 6 12 230 498 0 | 29.10.2020 09:28:01
11:30:00,910 DEEFR0169 | 0.84 1.21 1.21 1.28 | 7.253 | 8 88 88 88 0 | 29.10.2020 11:20:34
11:30:00,910 DEEFRK023 | 0.92 2.48 2.48 2.64 | 11.889 | 4 146 146 146 0 | 29.10.2020 11:22:30
11:30:00,910 DEEFRK035 | 0.00 0.08 0.08 0.13 | 0.102 | 0 2 2 2 0 | 29.10.2020 11:28:40
11:30:00,910 DEEFRK062 | 0.00 0.00 0.00 0.36 | 37.763 | 0 0 0 286 0 | 29.10.2020 08:36:27
11:30:00,910 PSI-MDT4.0 | 0.00 0.00 0.00 0.00 | 0.000 | 0 0 0 0 0 | 22.10.2020 11:10:30
11:30:00,912 -------------------------+--------------------------------+------------+----------------------------------------+---------------------
11:30:00,912 ALL | 100.00 100.00 100.00 100.00 | 605963.931 | 0 0 0 0 0 | 22.10.2020 11:10:19
11:30:00,912 =========================+================================+============+========================================+=====================
11:30:00,912
11:30:00,912 34 CLIENTS CONNECTED.
11:30:00,912
11:30:00,912 8=EVT TELEGRAMM RECV TskBrm->comsrv: |TskBrm|TskSls|TeleDb|
11:30:00,912 9=DEB BswTcpTskComSrv::passTele()
11:30:00,912 12=EVT | TELEGRAMM SEND comsrv->TskSls: |TskBrm|TskSls|TeleDb|
11:30:00,912 12=EVT | BswTcpTskComSrv::sendTeleTo:23 Bytes gesendet
Every line in that logfile is an event to sent to logstash and then transformed in some way. This works fine.
But what I want to archive is that the block between 11:30:00,909 LOAD STATISTICS and 11:30:00,912 34 CLIENTS CONNECTED. should be send as one message (as a side note: this statistic repeats every minute).
This is my filebeat config:
- type: log
enabled: true
paths:
- /fpsi_logs/*.comsrv.prot.*
fields:
fpsi_task: 'comsrv'
multiline.type: pattern
multiline.pattern: '^[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}\s\b[A-Z]{4}\b\s\b[A-Z]{10}'
multiline.negate: true
multiline.match: after
multiline.flush_pattern: '^[0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}\s[0-9]{1,3}\s[A-Z]{7}\s[A-Z]{9}'
What happens is, that all "Load Statistics" get send as separate messages but everything between two of them ist aggregated as one message as well.
At the moment I don't care of _grokparsefailures, just looking for a config to archive the result I need.
Someone here to help me?