Messages are not al same, how to grok?

Logs sent by my gateway are not exactly same, how to handle missing fields in the logs with grok.

Below Works

TDM,50c9676c6d24,udm- kernel: [1393127.673482] IN=br20 ***OUT=eth4*** MAC=74:ac:b9:1e:f3:00:e8:b1:fc:83:eb:68:08:00 SRC= DST= LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=6672 DF PROTO=TCP SPT=4638 DPT=443 WINDOW=513 RES=0x00 ACK URGP=0 MARK=0x4

Working Grok

%{IPORHOST:hostname},%{SYSLOGHOST},udm-%{BACULA_VERSION} kernel: \[%{MAVEN_VERSION}] IN=%{BACULA_DEVICE:incoming_int} OUT=%{BACULA_DEVICE:outgoing_int} MAC=%{MAC}:%{MAC}%{NOTSPACE} SRC=%{IPORHOST:ip.source} DST=%{IPORHOST:ip.destination} LEN=%{NUMBER:length} TOS=%{BASE16NUM:TOS} PREC=%{BASE16NUM:PREC} TTL=%{NUMBER:TTL} ID=%{NUMBER:ID} DF PROTO=%{WORD:Protocol} SPT=%{NUMBER:port.source} DPT=%{NUMBER:port.destination} WINDOW=%{NUMBER:window_size} RES=%{BASE16NUM} %{GREEDYDATA:flags}

non working logs with same grok above

TDM,50c9676c6d24,udm- kernel: [1394432.790629] IN=br0 OUT= MAC=76:ac:b9:1e:f2:fc:74:ac:b9:a8:83:73:08:00 SRC= DST= LEN=576 TOS=0x00 PREC=0x00 TTL=255 ID=43112 PROTO=TCP SPT=49958 DPT=8080 WINDOW=2144 RES=0x00 ACK PSH URGP=0

In the log above there is no value after OUT= and TCP flags in the end are different. How to make GROK compatible,

Instead of using grok I would suggest using dissect and kv. There is an example here.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.