Grok filter issue with data on Logstah config

Elastic Stack Logstash
1

Hi Team,
I am facing issue while using grok filter pattern for our log:
here is my log message:
2019-05-09 08:18:41.6499|INFO|Processor|8552|5825|Content|Calling for job id 5776, logid 17586

LogStash.config :

input {
beats {
port => 5044
}
}
filter {

mutate {
add_field => { "[@metadata][index]" => "%{[fields][type]}" }
}

	if [@metadata][index] == "imagelog"{

grok {
match => ["message","(?(([0-9]+)-)+ ([0-9]+:)+.*)|%{WORD:LOGLEVEL}|%{WORD:LOGSOURCE}|%{NUMBER:Id}|%{NUMBER:FileId}|%{DATA:Name}|%{GREEDYDATA:LOGMESSAGE}"]
}
}

	else
{

grok {
        match => ["message","(?<DATETIME>(([0-9]+)-*)+ ([0-9]+:*)+.*)\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{GREEDYDATA:LOGMESSAGE}"]
	}
	
}
	
}

In my case else condition working fine, but if condition not return required result
it return below result on kibana:
DATETIME field return: 2019-05-09 08:18:41.6499, 2019-05-09 08:18:41.6499|INFO|Processor|8552
LOGLEVEL field return: INFO, 5825
In LOGLEVEL I just want to return only INFO as per the logmessage.

2

If you have a message in which the fields are delimited I would recommend using dissect rather than grok.

dissect { mapping => { "message" => "%{ts}|%{loglevel}|%{logsource}|%{id}|%{fileid}|%{name}|%{message}" } }

If the entire set of events is pipe delimited you could also consider using a csv filter.

3

Thanks Badger,
Today its working fine without any changes.
I'll try with dissect filter also for better performance.

{ts} stand for timestamp?

4

Yes.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.