Multiple dates in log file defining the @timestamp date

Elastic Stack Logstash
1

Hi, I have a challenge, in the log-file of mine, I have three dates and would like to define the 3rd one as the @timestamp that is reflected in the ES and Kibana searches. What filter can I apply for this as the date one does not work?

Here is a example of the log-file:
Feb 29 03:15:26 172.29.58.134 804 <14>1 2016-02-29T03:15:30+02:00 10.10.10.12 1 - - - 2016-02-29T03:15:30.330+02:00 .........

Currently the imported date is the @timestamp for all the logs, I would like to get "2016-02-29T03:15:30.330+02:00 " as my timestamp

Thank you

2

Why does the date filter not work, it should. Perhaps providing your config would help.

3

HI, here is the filter:

`if "FLOWCREATE" in [tags] {
grok {
match => ["messages", "%{SYSLOGTIMESTAMP:Date1} %{SYSLOGHOST} %{CISCO_REASON}%{SYSLOGPROG} %{TIMESTAMP_ISO8601:Date2} %{IPORHOST} %{CISCO_REASON}- - - %{TIMESTAMP_ISO8601:Date3} %{HOSTNAME} RT_FLOW - RT_FLOW_SESSION_CREATE_LS %{SYSLOG5424PRINTASCII}: .* "]
add_field => ["received_on", "%{@timestamp}"]
add_field => ["received_frem", "%{host}"]
tag_on_failure => []
add_tag => "FLOWCREATE"
}

date {
match => ["Date3", "yyyy-MM-dd HH:mm:ss.SSSZ"]
}

mutate {
split => ["messages", " "]
}
kv{
field_split => " "
}
`

4

To match a timezone like +02:00, the correct pattern is ZZ to match offset with a colon
Z is matching offset without a colon
HTH

5

As your Date3 field is in ISO8601 format, you should be able to define the date filter like this:

  date {
    match => ["message", "ISO8601"]
  }

Note that the date filter always converts to UTC as this is required in Elasticsearch.

6

Thank you wiibaa and Christian_Dahlqvist for the feedback. I used both suggestions and it works now, thank you:
date { locale => "en" match => ["Date3", "yyyy-MM-dd HH:mm:ss.SSSZZ", "ISO8601"] timezone => "Africa/Windhoek" target => "@timestamp" add_field => { "debug" => "timestampMatched"} }