Trying to separate fields from logs using GROK

Elastic Stack Logstash
1

Hi am trying to separate fields from the log message below.

127.0.0.1 - - [25/Sep/2017:06:13:23 -0500] 4 "GET /CacheManager/HCD_CLIENTS/BN_HS HTTP/1.1" 200 118 ***:9090 "-" "-"

I have been trying to parse / separate the numeric 4 which is microsecond for average response time but doesn't seem to work below is the GROK filter i tried.

%{IPORHOST:clientip} %{USER:ident} %{USER:ident} \[%{HTTPDATE:timestamp}\] %{NOTSPACE:request} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)

2

Build your expression gradually. Start with the simplest possible, like %{IPORHOST:clientip} and add more and more until your done or until something breaks.

3

As @magnusbaeck pointed out, Build your expression gradually, I will recommend using GROK FILTER %{NUMBER} to read the numerics.

This should work :grinning:

%{IPORHOST:clientip} %{USER:ident} %{USER:ident} [%{HTTPDATE:timestamp}] %{NUMBER:responsetime} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)

4

Ya thanks, I have it now but my problem is when I apply this in logstash.conf file inside filter section the logs aren't appearing. It just stops.
This works fine while debugging in GROK debugger.

5

Have you looked for clues in the Logstash log?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.