Logstash Grok problem

Elastic Stack Logstash
1

Hello,
I'm using a grok filter for my logs. They come from severals folders and they have a common pattern i already manage to create :

grok {
        match => { "message" => "%{DATESTAMP:time}\|%{LOGLEVEL:Level}\|%{DATA:Logger}\|%{DATA:Message}\|%{DATA:Http_method}\|%{DATA:Http_url}\|%{DATA:Http_url_querystring}\|%{GREEDYDATA:Correlation_id}" }
    }

I used a lot of DATA or GREEDYDATA as i got pipe (|) to cut my logs. My PROBLEM here is that my Message field can be empty, filled with text or filled with text and a response time written like this :

"text text text 12.564ms text_or_empty"

I would like to create a more efficient filter for the Message field, by having two fields names Message like previously, and another one with the response time if it exists.

%{DATA:Message} would be
%{GREEDYDATA:Message} %{DATA:ResponseT}ms %{GREEDYDATA:Message}

So my problem is that when the ResponseT field does not exist, my entire log sentence is not parse to elasticsearch. I'm trying to solve this by improving my grok filter.
On Kibana i can make a filter taking only logs where "Message" got ms in his body, but this "ms" can exist in a word or have no ResponseT.

Have a great day, and thanks for your help !

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.