Remove trailing white space in logstash filter

Elastic Stack Logstash
1

I am trying to add filter for the below log format:

[24/Oct/2017 15:04:53 ] cluster WARNING Picking RM HA: ha

The filter I have added is:
[%{MONTHDAY:logDate}/%{MONTH:logMonth}/%{YEAR:logYear} %{TIME:logTime} ]%{SPACE}%{GREEDYDATA:platformType} +\s %{SPACE}%{LOGLEVEL:logLevel}%{SPACE}%{GREEDYDATA:message}

Issue is for platformType - cluster has trailing space and I would like to remove. Below is the screenshot from https://grokdebug.herokuapp.com/

Can any one help me in fetching only the platformType value by removing trailing spaces.

Thanks in Advance.

grok
grok.png1366×768 46 KB

2

What are the possible values of platformType ? If all values are only a single word, you'd be better off using something like NOTSPACE instead of GREEDYDATA (since greedydata is a bit expensive). Also, you should escape those brackets in your pattern.

This should work:

\[%{MONTHDAY:logDate}/%{MONTH:logMonth}/%{YEAR:logYear} %{TIME:logTime} \] %{GREEDYDATA:platformType} %{LOGLEVEL:logLevel} %{GREEDYDATA:message}
3

Sorry,
This did not work for me. A clarification I have copy pasted the log while logging this and there are 6 spaces between the words cluster and WARNING and these were removed on submission.

Can you try by explicitely adding 6 spaces between cluster and WARNING in the below log message before trying in https://grokdebug.herokuapp.com/.

[24/Oct/2017 15:04:53 ] cluster WARNING Picking RM HA: ha

Sorry for this.

Thanks in advance.

4

Ah, now I see the issue appearing indeed. This should work

\[%{MONTHDAY:logDate}/%{MONTH:logMonth}/%{YEAR:logYear} %{TIME:logTime} \] %{NOTSPACE:platformType}\s+%{LOGLEVEL:logLevel} %{GREEDYDATA:message}

Also, you can always remove leading/trailing whitespace post-grok with the strip option of the mutate filter, if you want to avoid trying to track down all possible corner cases for grok patterns.

5

Thanks a lot. It worked with NOTSPACE as suggested by you. However, I tried strip and was unsuccessful. Is there any snippet for this same trail space removal.

6

This should work if you place it after your grok filter.

mutate { 
  strip => ["platformType"] 
}
7

Thanks a lot. This worked for me.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.