Grok custom filters

Elastic Stack Logstash
1

Hi
I'm creating a custom filter for the log that I must store into the database.
It work (more or less) but there are still a couple of problem (testing it on Grok Debugger)

www.test.example.com 132.168.192.123 TLSv1.2 EABDE-IIA-AES128-GMM-SHDF434 - - 31/03/2016 05:39:47 CEST VvycA8ErsYMAAClUQhwAAABH - HTTP/1.1 GET 0 /test.txt "" 404 295 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 415 + 661 3860

my filter

%{IPORHOST:vhost}\s*%{IPORHOST:clientip}\s*%{PROG:sslProt}\s*%{PROG:chiper}\s*%{PROG:sslClient}\s*%{USER:auth}\s*%{TIMESTAMP_CUSTOM:timestamp}\s*%{PROG:uniqueID}\s*%{PROG:jSession}\s*(?:HTTP/%{NUMBER:httpversion})\s*%{WORD:request}\s*%{NUMBER:keepAlive}\s*%{PROG:path}\s*%{QS:query}\s*%{NUMBER:status}\s*%{NUMBER:rBytes}\s*%{QS:header}\s*%{QS:referer}\s*%{NUMBER:response}\s*%{DATA:connection}\s*%{NUMBER:bRecived}\s*%{NUMBER:bSent}

custom timestamp

TIMESTAMP_CUSTOM %{MONTHDAY}/%{MONTHNUM}/%{YEAR} %{TIME}? %{TZ}

there are 2 problems

1 - My custom timestamp can't read the timezone (as test I have remove the CEST and the %{TZ})
2 - Using the QS I receive back the value as in the example

"query": [
[
""""
]
],
...
...
...
"header": [
[
""-""
]
],
"referer": [
[
""Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)""
]
],

the value are between 2 "" instead of only one. I have try to remove them using (?:"%{QS:query}") but as return I receive an error.

Can someone help me?

Thank you

2

Sorry if I disturb again, I just realize that my question is quite unclear.
First of all I resolved alone the double "" problem so now I have some trouble whit the timezone.

Now, What I would like to know is if logstash have an option that can convert automatically the timezone from time zone name (CEST in this example) in a numeric form (+0200).

Thank you again

3

Timezone names aren't unique so such a mapping would be ambiguous. For example, CST could mean either China Standard time (UTC+8) or Central Standard Time (UTC-6). If you know which set of timezone names can occur in your logs you can use a translate filter to do the conversion, but you'd have to maintain the mapping table yourself.

4

Many thanks for your reply, I really appreciate.

Have a nice day.