Logstash Timstamp:_dateparsefailure issue

Elastic Stack Logstash
1

Hi,
I need to replace my log timestamp with logstash @timestamp.I have tried all possible ways and time formats but still not able to replace it and getting dateparsefailure issue every time.
Can anybody help me to sort out this?
Thanks in advance.

One line of my log sample is:
[ERROR@[140598203889408]2018-01-23 06:43:48.798511 in src/TMSInfo.cpp(1699)]TMSInfo::GetCorrelationKey: CIN is NULL failed to create corr key based on cin

My pipeline:

input {
file {
path => [""x:\x\x\x.log""]
start_position => beginning
}
}

filter {
grok {
match => { "message" => "%{GREEDYDATA}%{TIMESTAMP_ISO8601:msg} %{GREEDYDATA}"}
}
date {
locale => "en"
match => [ "msg", "yyyy-MM-dd HH:mm:ss.S" ,"ISO8601" , " yyyy-MM-dd HH:mm:ss" , "yyyy-MM-dd HH:mm:ss.SSSSS" , "yyyy-mm-dd hh:mm:ss.S" ]
target => "@timestamp"
add_field => { "debug" => "timestampMatched"}
}
}

output {
stdout { codec => rubydebug }
}

2

Did you try a date pattern ending with SSSSSS to capture the microseconds? I'm not sure that's supported so if it doesn't work you might have to cut them off with a mutate filter. What's printed by the stdout { codec => rubydebug } output?

3

Hi,I tried,but did not work out.
The output is like as below:
{
"path" => "x:\x\x\x.log",
"@timestamp" => 2018-04-09T12:08:27.328Z,
"@version" => "1",
"host" => "CT-IL0004",
"message" => "",
"tags" => [
[0] "_grokparsefailure"
]
}

4

But in this case the message was empty.

5

Take this output entry:
{
"path" => "x:\x\x\x.log",
"@timestamp" => 2018-04-09T12:13:03.191Z,
"@version" => "1",
"host" => "CT-IL0004",
"message" => " 2018-01-23 00:05:06.807734 : 1100023::CtRddmSocketDataSender::Connect: Re-initiating connection",
"tags" => [
[0] "_grokparsefailure",
[1] "_dateparsefailure"
]
}

6

Or take this output in which I did not remove "msg" log time stamp.The date format is same here:

{
"msg" => "18-01-23 00:08:57.443820",
"path" => "x:\x\x\x.log",
"@timestamp" => 2018-04-09T12:21:52.594Z,
"@version" => "1",
"host" => "CT-IL0004",
"message" => " 2018-01-23 00:08:57.443820 : 1100023::CtRddmSocketDataSender::Connect: Connect error 113 ",
"tags" => [
[0] "_dateparsefailure"
]
}

7

Your grok cuts off the start of the timestamp, so it's actually 'yy', but Magnus' 'SSSSSS' suggestion works fine for me. The pattern "yy-MM-dd HH:mm:ss.SSSSSS" results in:

       "msg" => "18-01-23 06:43:48.798511",
"@timestamp" => 2018-01-23T05:43:48.798Z,
   "testmsg" => "[ERROR@[140598203889408]2018-01-23 06:43:48.798511 in src/TMSInfo.cpp(1699)]TMSInfo::GetCorrelationKey: CIN is NULL failed to create corr key based on cin"

and

       "msg" => "18-01-23 00:08:57.443820",
"@timestamp" => 2018-01-22T23:08:57.443Z,
   "testmsg" => " 2018-01-23 00:08:57.443820 : 1100023::CtRddmSocketDataSender::Connect: Connect error 113 "
8

Hi,
Thank you so much to both of you.Its working fine now and serving my requirement.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.