How to override logstash @timestamp value with my date time field from the logs

Elastic Stack Logstash
1

My logs

Dec 12 13:45:43 [0x80000001][xsltmsg][notice]|6583|3018|0x00000000-0x00000000|235|
Dec 12 14:45:43 [0x80000001][xsltmsg][notice]|6583|3018|0x00000000-0x00000000|205|

My logstash config
filter {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:Call_Time} %{WORD:ApplicationName} }
}
date {
match => [ "Call_Time", "yyyy-MM-dd HH:mm:ss,SSS" ]
}
}

In output I can see Call_Time value is Dec 12 13:45:43

but since old logs needs to be mapped with the time when it was actually written instead loading time because kibana shows when you upload the logs that is why trying to override @timestamp value with the log creation date time

Also it would be great if someone can help me how to interpret _dateparsefailure tag.. i am seeing this on my console but don't know where to see to know exactly what went wrong while parsing date.

please execuse my less knowledge , i am new here.

2

Date-Filter expects a pattern matching your timeformat in the match-part.

Your pattern yyyy-MM-dd HH:mm:ss,SSS can't match your timestamp Dec 12 13:45:43.

The pattern you need should look like the following:
match => [ "Call_Time", "MMM dd HH:mm:ss" ]

but: You don't have a year in your logfile, so Logstash will default to the year when the Logstash process started. More information here Syslog date without year.

1 Like
3

Awesome ... Too good.... Quickest help....

It solved my problem.... Thank you so much

4

What about timezone? My logs are in US CST Timezone.

5

You don't have a year in your logfile, so Logstash will default to the year when the Logstash process started.

That's only true for old versions of the date filter. The behavior is better as of (I think) v2.1.0 of the filter.

6

What about timezone? My logs are in US CST Timezone.

As documented Logstash defaults to the computer's timezone. If the logs from a different timezone use the date filter's timezone option. The resulting timestamp in the @timestamp will always be UTC.

7

Thank you Magnus

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.