GROK Pattern creation

Elastic Stack Logstash
1

Hi all,

Need help in creating grok pattern that works for both the following type of logs

01/25-05:17:51.314622 192.168.1.1:138 -> 192.168.1.255:138
UDP TTL:64 TOS:0x0 ID:50222 IpLen:20 DgmLen:229 DF
Len: 201
=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

01/25-05:18:14.789246 192.168.1.20:33581 -> 192.168.1.1:80
TCP TTL:64 TOS:0x0 ID:25670 IpLen:20 DgmLen:60 DF
******S* Seq: 0x96391DAB  Ack: 0x0  Win: 0xFAF0  TcpLen: 40
TCP Options (5) => MSS: 1460 SackOK TS: 1563007755 0 NOP WS: 7

Thanks in advance
Anushka

3

Are those snort logs? Looks like this would be at least a good start integrations/plaintext.yml at main · elastic/integrations · GitHub

4

Are you reading logs with Filebeat then send to Logstash or only with Logstash?

5

This is from the agent integration which is essentially filebeat straight to elastic

6

You can use Grok contructor. The multiline pattern should be used in FB, and LS will receive a message as a single line.

01/25-05:17:51.314622 192.168.1.1:138 -> 192.168.1.255:138 UDP TTL:64 TOS:0x0 ID:50222 IpLen:20 DgmLen:229 DF Len: 201

(?<timestamp>%{MONTHNUM:month}/%{MONTHDAY:day}-%{TIME:time})%{SPACE}%{IP:sourceip}:%{POSINT:sourceport}%{SPACE}->%{SPACE}%{IP:destip}:%{POSINT:destport} %{WORD:protocol}%{SPACE}%{WORD}:%{POSINT:ttl}%{SPACE}%{WORD}:%{BASE16NUM:tos}%{SPACE}%{WORD}:%{POSINT:id}%{SPACE}%{WORD}:%{POSINT:iplen}%{SPACE}%{WORD}:%{POSINT:dgmlen}%{SPACE}%{WORD:flag}%{SPACE}%{WORD}: %{POSINT:len}

Line1
Line1891×416 9.45 KB

01/25-05:18:14.789246 192.168.1.20:33581 -> 192.168.1.1:80 TCP TTL:64 TOS:0x0 ID:25670 IpLen:20 DgmLen:60 DF ******S* Seq: 0x96391DAB Ack: 0x0 Win: 0xFAF0 TcpLen: 40 TCP Options (5) => MSS: 1460 SackOK TS: 1563007755 0 NOP WS: 7

(?<timestamp>%{MONTHNUM}/%{MONTHDAY}-%{TIME})%{SPACE}%{IP:sourceip}:%{POSINT:sourceport}%{SPACE}->%{SPACE}%{IP:destip}:%{POSINT:destport} %{WORD:protocol}%{SPACE}%{WORD}:%{POSINT:ttl}%{SPACE}%{WORD}:%{BASE16NUM:tos}%{SPACE}%{WORD}:%{POSINT:id}%{SPACE}%{WORD}:%{POSINT:iplen}%{SPACE}%{WORD}:%{POSINT:dgmlen}%{SPACE}%{WORD:flag}%{SPACE}%{DATA:something}%{SPACE}%{WORD}:%{SPACE}%{BASE16NUM:seq}%{SPACE}%{WORD}:%{SPACE}%{BASE16NUM:ack}%{SPACE}%{WORD}:%{SPACE}%{BASE16NUM:win}%{SPACE}%{WORD}:%{SPACE}%{POSINT:tcplen}%{SPACE}TCP Options \(%{POSINT:tcpoption}\)%{SPACE}=>%{SPACE}%{WORD}:%{SPACE}%{POSINT:mss}%{SPACE}%{WORD:sack}%{SPACE}TS: %{POSINT:ts}%{SPACE}%{INT:xvalue}%{SPACE}%{WORD:nop}%{SPACE}%{WORD}:%{SPACE}%{POSINT:ws}

Line2
Line21081×581 15.3 KB

7

Oh yes, this is very much helpful thank you!

8

This works! thanks!

9

Review the field names like something, sack, nop, ... and do you need timestamp only or all parts:
(?%{MONTHNUM:month}/%{MONTHDAY:day}-%{TIME:time}). Te 2nd grok pattern has only the timestamp field. Test, and correct if some filed has wrong type, like WORD expects only [a-zA-Z0-9_]
If everything is OK, you shouln't have the _grokparsefailure tag

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.