Failed to parse Timestamp

Elastic Stack Logstash
1

Hello Guys,

need some help.
In my logstash I am reading a date pattern from input files.
I have defined a format in my mapping with dd/MM/yyyy HH:mm:ss.SSS, so when it reads the input files it would follow the format i defined in mapping.

Now older files have a different format, where in old files have no milisecond(SSS) included in them. Reading around the discussion, it is said that Logstash should automatically fill up the missing milisecondds. Yet they dont, instead logstash throws a mapper_parsing_exception. As seen in the image below:

746×71 22.1 KB

I tried appending data when it encounters a dateparsefailure, like below:

 date {
    match => [ "tslice", "dd/MM/yyyy HH:mm:ss "]    
 }
 
 if ([tags] == "_dateparsefailure"){
    mutate {
        update => {"tslice" => "%{tslice}.000" }
    }
 }

So when it encounters the date, it would append zeroes which would match the dateformat defined in my mapping.

How do I make it that it will accept the input with no milisecond, and replace it with zeroes instead.
Any advice would be greatly appreciated! :smiley:

2

Why not just use the date filter to parse the date into ISO8601 format (including milliseconds) and use a mapping that supports ISO8601?

But if you insist on the current mapping, I suggest this to append .000 if there are no milliseconds present:

if [tslice] !~ /\.[0-9]+$/ {
  mutate {
    update => {
      "tslice" => "%{tslice}.000"
    }
  }
}
1 Like
3

Hello Magnus Baeck,

Thank you the solution worked perfectly. Correct me if I am wrong, the pattern of "/.[0-9]+$/" is a regular expression?

I see it has a pattern, if it sees a . a series of number mus follow. Is it correct?

4

Thank you the solution worked perfectly. Correct me if I am wrong, the pattern of "/.[0-9]+$/" is a regular expression?

Yes.

I see it has a pattern, if it sees a . a series of number mus follow. Is it correct?

Yes. The expression matches if the tslice fields ends with one or more digits, preceded by a period.

1 Like