Remove extra spaces between log messages using mutate & gsub

Elastic Stack Logstash
1

Log file Example:

INFO | jvm 1 | 2015/10/19 16:43:35 | logs logs logs text text text
INFO | jvm 1 | 2015/10/19 16:43:35 | logs logs logs text text text query to execute(in millis) : 28
INFO | jvm 1 | 2015/10/19 16:43:35 | Time taken by logs logs logs text text textexecute(in millis) : 336
INFO | jvm 1 | 2015/10/19 16:43:36 | Time taken by 'logs logs logs text text text' query to execute(in millis) : 1122

Logstash Config Used:
input {
stdin {
}
}

filter{
mutate {
gsub => ["message","|"," "]
}
grok{
patterns_dir => "C:\Docs\elk\logstash-1.5.4\patterns"
match => [ "message", "%{WRAPPERLOG}" ]
}
date {
match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}

output{
stdout{
codec => rubydebug
}
}

Grok Patterns Being Developed:
WRAPPERLOG %{LOGLEVEL:level} %{JVMNAME:jvmname} %{WRAPPER_DATESTAMP:timestamp} %{GREEDYDATA:logmessage}

JVMNAME %{WORD:jvm} %{INT:number}

WRAPPER_DATESTAMP 20%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND})

I am getting below grokparsefailure message when I tried multiple times, this is my first time using ELK any help is highly appreciated !!

In my config I have gsub for pipe but I am not able to find any gsub for multiple spaces between the log messages, I think that is my problem

2

Why use gsub in the first place? Why not include \| in the grok pattern? Use + to denote "at least one occurrence of the preceding token" to handle the variable number of spaces.

WRAPPERLOG %{LOGLEVEL:level} +\| +%{JVMNAME:jvmname} +\| +%{WRAPPER_DATESTAMP:timestamp} +\| +%{GREEDYDATA:logmessage}
2 Likes
3

Thanks a lot Magnus ! That was great help, I made the changes suggested by you and removed gsub and boom ! it worked !!