Extract Month and Year from date field

Elastic Stack Logstash
1

I am trying to extract Month from date field. But following is actually extracting month from @timestamp and not from first

input {
   generator {
      message => '{"first_report": "2019-05-30 14:57:59.11"}'
      count => 1
   }
}

filter {
   json { source => "message" }
   date { match => ["first_report", "yyyy-MM-dd HH:mm:ss.SS"]
        target => "first_report"
        add_field => {"month" => "%{+MM}" }
   }
   mutate { remove_field => ["message","path","host"] }
}

Output looks like this. I need month = 05

{
"month" => "11",
"@version" => "1",
"@timestamp" => 2019-11-22T16:07:57.768Z,
"first_report" => 2019-05-30T19:57:59.110Z,
"sequence" => 0
}

1 Like
2

sprintf date references always use @timestamp. Why not extract the month with grok?

3

Did lot of search and found that this can only be done with grok filter but I never use this filter so I am trying this way but giving me following error. any idea?

filter {
   json { source => "message" }
   date { match => ["first_report", "yyyy-MM-dd HH:mm:ss.SS"]
        target => "first_report"
   }
   grok {
      match => { "first_report" => "%{YEAR}-%{MONTH}-%{DAY} %{HOUR}:%{MINUTE}:%{SECOND}.%{SS}" }
   }

Getting following error.
exception=>#<Grok::PatternError: pattern %{SS} not defined>,

4

@Badger,
can you help me find what this error is on grok, how to write this.

5

You do not have to match the entire field

grok { match => { "first_report" => "^%{YEAR}-%{MONTH:month}" } }
2 Likes
6

use try following three all failed

grok { match => { "first_report" => "^%{YEAR:year}-%{MONTH:month}" } }

and as you posted

grok { match => { "first_report" => "^%{YEAR}-%{MONTH:month}" } }

and

grok { match => { "first_report" => "^%{YEAR}-%{MONTH}" } }

All of them failed
{
"@timestamp" => 2019-11-22T17:51:39.840Z,
"message" => "{"first_report": "2019-05-30 14:57:59.11"}",
"@version" => "1",
tags" => [
[0] "_grokparsefailure"
],
"sequence" => 0,
"first_report" => 2019-05-30T19:57:59.110Z
}

7

fixed. found it

found this page which has fixed pattern name
https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns

grok { match => { "first_report" => "^%{YEAR:year}-%{MONTHNUM:month}" } }

2 Likes
8

Thank you @Badger

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.