Help with grok pattern Logstash

Elastic Stack Logstash
1

I need help with creating a grok pattern to match the following data and create them as new fields. I have attempted to use the Grok debugger online and it matches my custom patterns but when I apply this in Logstash it does not work. I know it's probably something simple I have missed so any help to point it out is appreciated.

The lines of messages I want to create fields on:
EQID01 - 14.05.2020 08:19:48 - Reply from 192.168.1.2: bytes=32 time=90ms TTL=255
EQID01 - 14.05.2020 08:19:49 - Reply from 192.168.1.2: bytes=32 time=1ms TTL=255
EQID01 - 14.05.2020 08:19:50 - Reply from 192.168.1.2: bytes=32 time=1ms TTL=255
EQID02 - 14.05.2020 08:19:51 - Reply from 192.168.1.3: bytes=32 time=2ms TTL=255
EQID02 - 14.05.2020 08:19:52 - Reply from 192.168.1.3: bytes=32 time=150ms TTL=255

The custom patterns:

EQ_NAME [a-zA-Z0-9._-]+
CUSTOM_TIME (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[./-](?:0?[1-9]|1[0-2])[./-](?>\d\d){1,2}[- ](?!<[0-9])(?:2[0123]|[01]?[0-9]):(?:[0-5][0-9])(?::(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))(?![0-9])
REPLY_FROM (?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9])
BYTES ([0-9][0-9]{1,3})
MILLI_SECONDS ([0-9][0-9]{1,4})
TTL ([0-9][0-9]{1,4})

The grok pattern:

%{EQ_NAME:equipment}...%{CUSTOM_TIME:timestamp}.*%{REPLY_FROM:reply_from}.+?%{BYTES:bytes}.+?%{MILLI_SECONDS:delay}.+?%{TTL:ttl}
2

Hi.
If the fields do not change, you can try this:

^%{DATA:host.name} - %{DATA:@timestamp} - %{DATA:event.type} %{IP:destination.ip}: bytes=%{INT:bytes} time=%{DATA:time} TTL=%{INT:ttl}$

Produces:
{
"@timestamp": "14.05.2020 08:19:52",
"bytes": "32",
"host": {
"name": "EQID02"
},
"destination": {
"ip": "192.168.1.3"
},
"time": "150ms",
"event": {
"type": "Reply from"
},
"ttl": "255"
}

3

Thanks for that.

I tried it and it was the same result. For some reason I do not see the new fields in the index management in Kibana even if I refresh it.

My grok section looks like this if this is correct?

grok {
          match => ["message", "^%{DATA:host.name} - %{DATA:@timestamp} - %{DATA:event.type} %{IP:destination.ip}: bytes=%{INT:bytes} time=%{DATA:time} TTL=%{INT:ttl}$"]
}
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.