Remain json data as field in Logstash


(Ducheol Kim) #1

Hi Guys.

Currently, I'm building component which log json data in ES.
So I'm trying to use below grok filter pattern to parse , but it isn't work.

Input
Tue, 06 Oct 2015 20:04:15 +0000 - http://localhost/testurl?id=127 - {"request":{"TEST":"1234"}} - 200-{"response":{"status":"OK","count"}

grok config
grok{
patterns_dir => "./patterns"
match => {
"message" => "%{DATESTAMP_RFC2822:time} - %{URI:uri} - {DATA:request} - {POSINT:code}- {DATA:response}"
}
}

What is wrong ? It is about json or about others ?


(Lee Carter) #2

new here so could be way off but looks like your pattern "DATESTAMP_RFC2822" doesn't match your log timestamp format... in the patterns file it says:

DATESTAMP_RFC822 %{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}

your log format looks to be (DAY, MONTHDAY, MONTH, YEAR, TIME, TZ...)

you could try this tool to build the grok statement... been helpful for me on what little I've done.

https://grokdebug.herokuapp.com/


(Ducheol Kim) #3

Thanks Lee.

But, according to below page, DATESTAMP_RFC2822 is like below.

DATESTAMP_RFC2822 %{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601_TIMEZONE}
Tue, 06 Oct 2015 20:04:15 +0000

Looks like DATESTAMP_RFC2822 is same with my date format. isn't it ?

Regards
Ducheol


(Ducheol Kim) #4

Sorry. It is my mistake.

I didn't put % before the pattern. It work fine after put the %

Thanks


(system) #5