Date parsing issue

[rajsolanki (rajesh)]

I m having strange issue with Date parsing. Here is my config.

input { stdin { } }
output { stdout { codec => rubydebug } }
filter {
  date {
    match => ["message", "ISO8601"]
  }
}

when i run echo '2022-11-17 15:30:32.000' | /bin/logstash -f date.config output is

"@timestamp" => 2022-11-17T21:30:32Z,
"event" => {
"original" => "2022-11-17 15:30:32.000"
}

You can see @timestamp is removing 000

when i run echo '2022-11-17 15:30:32.001' | /bin/logstash -f date.config output is

"@timestamp" => 2022-11-17T21:30:32.001Z,
"event" => {
"original" => "2022-11-17 15:30:32.001"
},

So any way to force logstash to assign @timestamp including 000 ?

[Rios]

filter {
  date {
    match => ["message", "YYYY-MM-dd HH:mm:ss.SSS"]
	target => "time"
  }
}

{
    "@timestamp" => 2022-11-17T21:23:09.258725600Z,
          "time" => 2022-11-17T14:30:32.000Z,
       "message" => "2022-11-17 15:30:32.000"
}

[Badger]

Why does it matter? Do you have a downstream system that consumes rubydebug output? The underlying Timestamp objects both have the same precision.

This PR suggests that Timestamp#to_string in logstash 7.x always had 3 digits of sub-second precision, and once Elastic ship a release containing that PR then logstash 8 will too.

[rajsolanki (rajesh)]

yes downstream system consumes that plus i have a pipe line which converts @timestamp value to localtime (CST) which is used in an alert email generated by watcher.

if(ctx.containsKey("@timestamp")) { 
            
            def sf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
            def dt1 = String.valueOf(ctx['@timestamp']);
            def dt = sf.parse(dt1);
            def calendar = sf.getCalendar();
            calendar.setTime(dt);
            def instant = calendar.toInstant();
            def localDateTime = LocalDateTime.ofInstant(instant, ZoneOffset.UTC);
            def localtime = localDateTime.minusHours(12);
            
            ctx.localtime =  localtime.toString() + ' CST';

          }
        

[rajsolanki (rajesh)]

@Badger

This PR suggests that Timestamp#to_string in logstash 7.x always had 3 digits of sub-second precision, and once Elastic ship a release containing that PR then logstash 8 will too.

So one of the label on link has 8.4 . does it mean its released as part of 8.4+ ?

[Badger]

Yes, it is labelled as 8.4.0 and it gets mentioned as the third entry in the Notable issues fixed section of the 8.4.0 release notes.