Ignore date offset in logstash

Elastic Stack Logstash
1

HI, I have the following date format:
"2017-12-01 04:39:34-04:000"

Currently, I use this configuration:

date{
                match => ["[documentsList][0][commandTimestamp]","yyyy-MM-dd HH:mm:ssZZ'0'"]
                target => ["command_timestamp"]
                remove_field => ["%{timestamp}"]
        }

Now I want to ignore the date offset, i.e. ignore the "-04:000".
When I try to change the configuration to:

date{
                match => ["[documentsList][0][commandTimestamp]","yyyy-MM-dd HH:mm:ss"]
                target => ["command_timestamp"]
                remove_field => ["%{timestamp}"]
        }

I get date parse failure.
Why is this happening? How can I fix it?

2

This might help: Want to capture timeStamp without milliseconds.

3

HI, Thanks for the answer.
I tried:

    mutate{
            copy => {"[documentsList][0][commandTimestamp]" => "[documentsList][0][noffset_commandTimestamp]"}

            gsub => ["[documentsList][0][noffset_commandTimestamp]","/.{8}$/g",""]
    }

    date{
            match => ["[documentsList][0][noffset_commandTimestamp]","yyyy-MM-dd HH:mm:ss"]
            target => ["no_offset_command_timestamp"]
    }

and the no offset timestamp remain unchanged.
I tried also in the gsub line:
/-\d{2}+:\d{3}/g

to no avial.
I tried also without the "/" on the start and "/g" in the end. Same result.

Any advice?

4

Don't assume that the options to a mutate filter execute in the order given. They always execute in a fixed order:

So, gsub runs before copy. Split your mutate filter in two. Secondly, the gsub expression is wrong. Try this:

gsub => ["[documentsList][0][noffset_commandTimestamp]", ".{8}$", ""]
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.