Grok filter pattern for nginx to parse request body

Elastic Stack Logstash
1

Hi,

i'm trying to parse the request body with logstash grok pattern for nginx generated logs.

below is my grok pattern-

"%{WORD:req_body}"

and i get the parsed field like below-

 "req_body" : "req_body",
 "@version" : "1"

i am not getting the request body being parsed. what am i missing here?
can someone please help me in this?

2

Please provide a sample message and the complete grok filter configuration.

3

here is my sample message

127.0.0.1 - - [28/Jul/2020:09:11:42 +0000] "POST /index1/type1/1 HTTP/1.1" 401 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/12.0.3245.52 Safari/537.36" "-" 0.007 req_body:"{"name" : "elastic"}"

i use below grok filter to parse out above log message

%{IP:clientip} - - \[%{HTTPDATE:timestamp}\] \"%{WORD:action} %{URIPATHPARAM:request} %{WORD:protocol}/%{NUMBER:protocolNum}\" %{NUMBER:response} %{NUMBER:bytes} \"-" %{QS:agent} \"-" %{BASE10NUM:request_duration} %{WORD:req_body}

4

I would expect you to have to quote all four double quotes.

If you match %{WORD:req_body} against req_body:"{"name" : "elastic"}" then I would expect you to get exactly what you got

 "req_body" : "req_body",

Did you mean to use %{GREEDYDATA:req_body} to capture the rest of the line?

5

Thanks @Badger. that request body is now coming as part of the GREEDYDATA message. but i have some more fields after the request_body field, some thing like "upstream_addr". this field is also included within the request body field.

i wanted to parse the request_body only within that field.

127.0.0.1 - - [28/Jul/2020:09:11:42 +0000] "POST /index1/type1/1 HTTP/1.1" 401 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/12.0.3245.52 Safari/537.36" "-" 0.007 req_body:"{"name" : "elastic"}" upstream_addr=127.0.0.1:9200

after your suggestion i changed the grok pattern to below:

%{IP:clientip} - - \[%{HTTPDATE:timestamp}\] \"%{WORD:action} %{URIPATHPARAM:request} %{WORD:protocol}/%{NUMBER:protocolNum}\" %{NUMBER:response} %{NUMBER:bytes} \"-" %{QS:agent} \"-" %{BASE10NUM:request_duration} %{GREEDYDATA:req_body} %{NUMBER:upstream_addr}

my parsed feilds are coming like below:

"req_body" : """
req_body:"POST /index1/type1/1 HTTP/1.1" upstream_addr=10.206.29.131:9200
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.