How to parse different time formats

Elastic Stack Logstash
1

Hello,

I use a file input with a multiline codec and a pattern that says if the line starts with a date then it is a new line.

input {
    file {
    ...
    codec => multiline {
    pattern => "^%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}?"
    negate => true
    what => previous

I also use a grok filter that extract the data from the date:

grok {
    match => { 
      "message" => "^%{MONTH:Month}%{SPACE}%{MONTHDAY:Day},%{SPACE}%{YEAR:Year}

This works only for dates in the format May 10, 2017 2:35:30 PM
But I have also dates in the format 2017-05-10 2:35:30 PM.

How can I include both in one config file?

2

You can combine both on one line! grok allows for an array of patterns, as one possible solution:

match => {
  "message" => ["%{PATTERN1}", "%{PATTERN2}"]
}

Where %{PATTERN1} could be your entire "^%{MONTH:Month}%{SPACE}%{MONTHDAY:Day},%{SPACE}%{YEAR:Year}, and %{PATTERN2} could be the other date pattern.

With multiple fields, the first match wins and suspends further matching. With that understanding, you should put the most common pattern first (e.g. if pattern1 occurs 70% of the time, and pattern2 the other 30%, then put pattern1 as the first pattern in the array).

1 Like
3

Thank you for your answer. It works really fine inside the grok filter.
But is it also possible for the multiline pattern in the file input?

4

You'd have to do some fancy work with (?: and | to make that work, effectively becoming an OR clause.

1 Like
5

Yes thank you. That works fine :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.