Hi Guys.
Currently, I'm building component which log json data in ES.
So I'm trying to use below grok filter pattern to parse , but it isn't work.
Input
Tue, 06 Oct 2015 20:04:15 +0000 - http://localhost/testurl?id=127 - {"request":{"TEST":"1234"}} - 200-{"response":{"status":"OK","count"}
grok config
grok{
patterns_dir => "./patterns"
match => {
"message" => "%{DATESTAMP_RFC2822:time} - %{URI:uri} - {DATA:request} - {POSINT:code}- {DATA:response}"
}
}
What is wrong ? It is about json or about others ?
l1carter
(Lee Carter)
October 7, 2015, 4:53pm
2
new here so could be way off but looks like your pattern "DATESTAMP_RFC2822" doesn't match your log timestamp format... in the patterns file it says:
DATESTAMP_RFC822 %{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}
your log format looks to be (DAY, MONTHDAY, MONTH, YEAR, TIME, TZ...)
you could try this tool to build the grok statement... been helpful for me on what little I've done.
https://grokdebug.herokuapp.com/
Thanks Lee.
But, according to below page, DATESTAMP_RFC2822 is like below.
USERNAME [a-zA-Z0-9._-]+
USER %{USERNAME}
EMAILLOCALPART [a-zA-Z][a-zA-Z0-9_.+-=:]+
EMAILADDRESS %{EMAILLOCALPART}@%{HOSTNAME}
INT (?:[+-]?(?:[0-9]+))
BASE10NUM (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))
NUMBER (?:%{BASE10NUM})
BASE16NUM (?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))
BASE16FLOAT \b(?<![0-9A-Fa-f.])(?:[+-]?(?:0x)?(?:(?:[0-9A-Fa-f]+(?:\.[0-9A-Fa-f]*)?)|(?:\.[0-9A-Fa-f]+)))\b
POSINT \b(?:[1-9][0-9]*)\b
NONNEGINT \b(?:[0-9]+)\b
WORD \b\w+\b
NOTSPACE \S+
SPACE \s*
DATA .*?
GREEDYDATA .*
QUOTEDSTRING (?>(?<!\\)(?>"(?>\\.|[^\\"]+)+"|""|(?>'(?>\\.|[^\\']+)+')|''|(?>`(?>\\.|[^\\`]+)+`)|``))
UUID [A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}
# URN, allowing use of RFC 2141 section 2.3 reserved characters
This file has been truncated. show original
DATESTAMP_RFC2822 %{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601_TIMEZONE}
Tue, 06 Oct 2015 20:04:15 +0000
Looks like DATESTAMP_RFC2822 is same with my date format. isn't it ?
Regards
Ducheol
Sorry. It is my mistake.
I didn't put % before the pattern. It work fine after put the %
Thanks