For a long time I assumed @timestamp was being parsed correctly; the time in the kibana Time field matches the @timestamp field, which matched the data being logged; however, logstash crashed over the weekend so I started it up this morning, and all the missed logs have @timestamp set to today, completely different to the time in the log.
I'm using the following pattern...
%{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer})(?:;|) %{QS:agent}
...which the grok debugger seems to be fine with (some data removed)...
{
"timestamp": [
[
"22/Nov/2015:08:00:43 +0100"
]
],
"MONTHDAY": [
[
"22"
]
],
"MONTH": [
[
"Nov"
]
],
"YEAR": [
[
"2015"
]
],
"TIME": [
[
"08:00:43"
]
],
"HOUR": [
[
"08"
]
],
"MINUTE": [
[
"00"
]
],
"SECOND": [
[
"43"
]
],
"INT": [
[
"+0100"
]
],
...when parsing the following...
x.x.x.x - - [22/Nov/2015:08:00:43 +0100] "GET /secret HTTP/1.1" 304 0 "" "Mozilla/5.0 (Windows NT 6.1; rv:42.0) Gecko/20100101 Firefox/42.0"