Can't replace timestamp , getting [1] "_dateparsefailure"

Elastic Stack Logstash
1

I am getting [1] "_dateparsefailure" when trying to replace timestamp

My timestamp is "@timestamp" => 2017-01-23T17:58:23.434Z

and I want to replace it by "GenerateTime" => "2017/23/01 12:58:23" from my input

so I am using below
date {
timezone => "America/New_York"
match => [ "GenerateTime", "YYYY/MM/dd HH:mm:ss" ]
target => "@timestamp"
}

but it gives me [1] "_dateparsefailure" ?
PS
Also I don't understand why timestamp" is 2017-01-23T17:58:23.434Z not 2017/23/01 12:58:23 , 5h difference

2

but it gives me [1] "_dateparsefailure" ?

Is your input really "2017/23/01 12:58:23"? If so it's YYYY/dd/MM and not YYYY/MM/dd.

Also I don't understand why timestamp" is 2017-01-23T17:58:23.434Z not 2017/23/01 12:58:23 , 5h difference

The @timestamp field is always UTC.

1 Like
3

Now it matches :slight_smile: I verified it multiple times, just haven't noticed the mistake ...Thnx again !!!

4

Sorry, one more thin, do you know why it is "@timestamp":"2017-01-23T05:19:49.000Z" so 05:19 instead of 17:19 ?

5

Sorry, one more thin, do you know why it is "@timestamp":"2017-01-23T05:19:49.000Z" so 05:19 instead of 17:19 ?

What's the exact input string and what date pattern are you using?

6

Input string is i.e. all dates like 2017/20/01 13:53:35 I am using bold one in above 'date' after 'csv'

<141>Nov 3 12:53:35 DC1. 1,2017/20/01 13:53:35,001901000999,THREAT,file,1,2017/20/01 13:53:35,1.1.1.1,2.2.2.2,0.0.0.0,0.0.0.0,RULE fake,me,,APP test,vsys1,app,rep,ae1.2,ae2.1,LF-elk,2017/20/01 13:53:35,33891243,1,445,66666,0,0,0x0,tcp,test,"WebFix.exe",Microsoft PE File(52060),any,low,server-to-client,5210010,0x0,10.0.0.0-10.255.255.255,10.0.0.0-10.255.255.255,0,,0,,,0,,,,,,,,0,38,0,0,0,PROD,DC1,

7

I asked two questions earlier. Please answer both. Please also answer the following:

  • What's the resulting @timestamp value for the example input you've provided?
  • What's the timezone of the machine running Logstash?
8

I did new test:

  1. from stdout I can see "@timestamp" => 2017-01-24T02:10:53.000Z
  2. from OS running logstash $ date
    Tue Jan 24 14:10:53 EST 2017
9

Please answer all questions I ask. The remaining one (last chance) is: What date pattern are you using?

10

Here it is , I have timezone commented out

date {
#timezone => "America/New_York"
match => [ "GenerateTime", "YYYY/dd/MM HH:mm:ss" ]
target => "@timestamp"
}

11

Okay. Please show the output of this command:

echo '2017/20/01 13:53:35' | logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } } filter { date { match => [ "message", "YYYY/dd/MM HH:mm:ss" ] } }'

(You may have to provide the full path to the Logstash binary.)

12

That one looks good :slight_smile: What's going on ?

{
"@timestamp" => 2017-01-20T13:53:35.000Z,
"@version" => "1",
"host" => "01b23f0c8ad0",
"message" => "2017/20/01 13:53:35"
}

13

Since 2017/20/01 13:53:35 is EST (right?) you need to specify the timezone so that @timestamp gets the correct UTC value, but yes, apart from the timezone offset things are okay.

14

I can see now that thing sending to my input uses 12h format:tired_face:

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.