Problem with date filter

Elastic Stack Logstash
1

Hello,
I have been trying to use the date filter plugin but without success. I am trying to parse a field and target it into @timestamp field.

My message field contains a date string like the following:

2022-05-09 09:19:07,900 [xJQsL/r76U:396] DEBUG

I am creating two new fields: modb.date and modb.ms. The field mobd.date and modb.ms are corretly parsed , and respectively get the values 2022-05-09 09:19:07 and 900. Then i concatenate both to create a new field called modb.time with the format 2022-05-09 09:19:07.900.

The problem happens when i try to use the date filter on this modb.time field. When the documents get indexed i can see on the field tags, "_dateparsefailure". I have tried to parse with multiple formats but without success.

I can only parse the modb.time value when i remove the miliseconds (.SSS) in the parsing format.

filter {
    dissect {
        mapping => { "message" => "%{[modb][date]},%{[modb][ms]}[" }
    }

    mutate {
        add_field => { "[modb][time]" => "%{[modb][date]}.%{[modb][ms]}" }
    }

    date {
        #match => [ "timestamp", "MM/d/yyyy h:mm:ss a" ]
        match => [ "[modb][time]", "yyyy-MM-dd hh:mm:ss.SSS" ]
        target => ["@timestamp"]
    }
}

Do you have any ideias why is this happening?
Regards

2

2022-05-09 09:19:07,900 [xJQsL/r76U:396] DEBUG

There is a space after the ms in timestamp and the [ that isn't account for in the Grok pattern.

Try this. %{[modb][date]},%{[modb][ms]} [

1 Like
3

Nice finding :sweat_smile: It was resolved with your solution.
Thank you!

1 Like
4

Seems like i am having issues again with the date filter. Yesterday was working fine, but today is not working anymore...

Field to be parsed by date filter: 2022-05-10 15:11:28.209

image
image910×39 2.4 KB

Filter used:

date {
        match => [ "[modb][time]", "yyyy-MM-dd hh:mm:ss.SSS" ]
        target => ["@timestamp"]
    }

I didn't touch the date filter since it was working.
Do you see anything that i am not seeing?

Regards

5

Try big H's for the hour. Documentation.

match => [ "[modb][time]", "yyyy-MM-dd HH:mm:ss.SSS" ]
3 Likes
6

Thanks Aaron, it is working. I was confused since with the previous pattern worked and suddenly stopped.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.