How to use multiline pattern matching grok filter?

Elastic Stack Logstash
1

My use case is to find the line which comes before some pattern line and if that line is found in
then dump in ES.

Here is a small example.
In this, if I am finding "dhd_bus_rxctl: rxcnt_timeout" line before the line "dhd_check_hang: Event HANG send up due to". For this how we can use grok filter.

15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] [] (kthread+0xe0/0xe4) from [] (ret_from_fork+0x14/0x20)
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
[15119.753902] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753917] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753937] dhd_prot_ioctl : bus is down. we have nothing to do
[15119.791431] [] (schedule_timeout+0x158/0x25c) from [] (0xea1e0000)
[15119.799331] kworker/3:2 R running 0 29597 2 0x00000000
[15119.805699] [] (__schedule+0x3d0/0x8a4) from [] (worker_thread+0x1fc/0x3dc)
[15119.814384] [] (worker_thread+0x1fc/0x3dc) from [] (kthread+0xe0/0xe4)
[15119.822637] [] (kthread+0xe0/0xe4) from [] (ret_from_fork+0x14/0x20)
[15119.830710] kworker/u8:1 S c0ab2fd4 0 29738 2 0x00000000
[15119.837078] [] (__schedule+0x3d0/0x8a4) from [] (worker_thread+0x1fc/0x3dc)
[15119.845763] [] (worker_thread+0x1fc/0x3dc) from [] (kthread+0xe0/0xe4)
[15119.854015] [] (kthread+0xe0/0xe4) from [] (ret_from_fork+0x14/0x20)
[15119.862088] kworker/u8:4 S c0ab2fd4 0 29739 2 0x00000000
[15119.868455] [] (__schedule+0x3d0/0x8a4) from [] (worker_thread+0x1fc/0x3dc)
[15119.877140] [] (worker_thread+0x1fc/0x3dc) from [] (kthread+0xe0/0xe4)
[15119.885391] [] (kthread+0xe0/0xe4) from [] (ret_from_fork+0x14/0x20)
[15119.893468] Sched Debug Version: v0.10, 3.10.96+ #1
[15119.898337] ktime

2

@MariumHassan

3

Does this help you?

(?<the_line>.*)\n.*dhd_check_hang: Event HANG

4

@Jenni, Yeah its working but it is adding one extra tag i.e. "rxcnt_timeout" in output json.
Shown as follow.

" rxcnt_timeout": "rxctl: rxcnt_timeout=5, rxlen=0\n 15119.753889] [] (kthread+0xe0/0xe4) from [] (ret_from_fork+0x14/0x20)\n 15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0\n 15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0\n 15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0\n 15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0\n [15119.753902] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2",

5

Slightly change in use case, if line "dhd_check_hang: Event HANG send up due to" is matched in logs and line should not "dhd_bus_rxctl: rxcnt_timeout" matched in logs then only dump in ES

6

it matches all lines above "dhd_check_hang: Event HANG"

need to match "dhd_bus_rxctl: rxcnt_timeout" before/after "dhd_check_hang: Event HANG"

I am editing input as

[1202095.534161] init: smbd main process (15913) killed by TERM signal
[1211904.223712] init: smbd main process (22226) killed by TERM signal
[1220472.750723] systemd-hostnamed[14939]: Warning: nss-myhostname is not installed. Changing the local hostname might make it unresolveable. Please install nss-myhostname!
[1222858.122914] init: smbd main process (3976) killed by TERM signal
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] (kthread+0xe0/0xe4) from (ret_from_fork+0x14/0x20)
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
[15119.753902] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753917] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753937] dhd_prot_ioctl : bus is down. we have nothing to do
[15119.791431] (schedule_timeout+0x158/0x25c) from (0xea1e0000)
[15119.799331] kworker/3:2 R running 0 29597 2 0x00000000
[15119.805699] (__schedule+0x3d0/0x8a4) from (worker_thread+0x1fc/0x3dc)
[15119.814384] (worker_thread+0x1fc/0x3dc) from (kthread+0xe0/0xe4)
[15119.822637] (kthread+0xe0/0xe4) from (ret_from_fork+0x14/0x20)
[15119.830710] kworker/u8:1 S c0ab2fd4 0 29738 2 0x00000000
[15119.837078] (__schedule+0x3d0/0x8a4) from (worker_thread+0x1fc/0x3dc)
[15119.845763] (worker_thread+0x1fc/0x3dc) from (kthread+0xe0/0xe4)
[15119.854015] (kthread+0xe0/0xe4) from (ret_from_fork+0x14/0x20)
[15119.862088] kworker/u8:4 S c0ab2fd4 0 29739 2 0x00000000
[15119.868455] (__schedule+0x3d0/0x8a4) from (worker_thread+0x1fc/0x3dc)
[15119.877140] (worker_thread+0x1fc/0x3dc) from (kthread+0xe0/0xe4)
[15119.885391] (kthread+0xe0/0xe4) from (ret_from_fork+0x14/0x20)
[15119.893468] Sched Debug Version: v0.10, 3.10.96+ #1
[15119.898337] ktime

output should be matched lines.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.