Parse string to date and have difference between 2 dates

Elastic Stack Logstash
1

Hi !
I have a string with this format :

Fri Feb 16 14:30:09 CET 2018

I init a field like this :

mutate {
add_field => { "dateLog" => "%{message}" }
}
mutate {
gsub => [ "dateLog", "INFOS:\s", "" ]
}
mutate {
gsub => [ "dateLog", "\s([0-9.])\s([A-Z])\s([^ ])(\s)([^ ])\s([^ ])\s([^ ]\s[A-Za-z]\s[0-9]\s[0-9:]\s[A-Z]\s[0-9])\s([0-9]{3})\s([0-9.-])\s([0-9.-])\s[A-Za-z0-9-]\s[^ ]\s[^ ]\s[^ ]\s", "" ]
}

The string format is ok, no problem with this but I need to convert it to date, and have the difference with @timestamp.

How can I do ?

Do you have an exemple for me ?

2

Have you looked into the date filter to convert the string into a date more simply than your regex?
https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html

From there, I think you could use the elapsed plugin to find the time difference you want. https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-elapsed.html

3

thank you for your reply : I tried this but it does not work :
date {
match => [ "dateLog", "MMM d H:mm:ss yyyy" ]
}

4

That does not match "MMM d H:mm:ss yyyy". There is a day of the week (EEE) and an identity timezone (ZZZ) in there.

5

I used this recently at a customer, and it might help point you in the right direction...

# Convert the dates to a proper timestamp.
date {
  match => [ "BEGIN", "dd/MM/YYYY HH:mm:ss.SSSSSS" ]
  timezone => "UTC"
  target => "BEGIN"
}
date {
  match => [ "LAST", "dd/MM/YYYY HH:mm:ss.SSSSSS" ]
  timezone => "UTC"
  target => "LAST"
}

# If the dates parsed properly calculate the runtime in seconds (precision is ms).
if "_dateparsefailure" not in [tags] {
  ruby {
    code => "
      event.set('[RUNTIME_SEC]', (event.get('[LAST]').to_f - event.get('[BEGIN]').to_f).round(3));
    "
  }
}
6

So how can I do ?

7

Thank you for your reply I will test it when I succeed to parse my string

8

That would match if dateLog were something like "Feb 16 14:30:09 2018", but you said it is formatted as "Fri Feb 16 14:30:09 CET 2018". So you need to modify the pattern to include items to match the "Fri" and "CET". The pattern has to match every part of the date string.

9 
     "dateLog" => "Feb 16 14:30:09 2018",

I have change the format like you say, but nothing has changed .... I do not have a field date

10

So what exactly does your configuration look like? If I run logstash with this configuration

input { stdin { } }
output { stdout { codec => rubydebug } }
filter { date { match => [ "message", "MMM d H:mm:ss yyyy" ] } }

and feed it Feb 16 14:30:09 2018, then it gets parsed...

    "@timestamp" => 2018-02-16T19:30:09.000Z,
       "message" => "Feb 16 14:30:09 2018",
11

Yes but how can I inject :
"message" => "Feb 16 14:30:09 2018",
in date format inside ElasticSearch ?

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.