Help with Grok patterns to parse date and name index from path

Elastic Stack Logstash
1

Hi, I kindly ask you some help with these Grok patterns.
I have two requirements: the first is to extract the date from path and use it to name the Elasticsearch index. The second is to parse a date with a strange format.

Given that the path is this: /usr/share/logstash/logs/file_report_20180726_0730.csv
I want to name the index logstash-2018.07.26

In the filter section, I have used this Grok filter:

grok {
    match => { "path" => %{GREEDYDATA}/file_report_%{INT:file_date}_%{INT}.csv }
}

But this produces file_date = 20180726, while I would like to have file_date = 2018.07.26 in order to do as follows in the ouput section:

output {
    elasticsearch {
	    hosts => ["elasticsearch:9200"]
	    index => "logstash-%{file_date}"
    }
}

Moreover, I have a date field which I need to parse. This field is formatted like this: 2018-07-08 09:49:43.868+02

Edit: one more question. Is it possible to match a date even if two events could have a different format? For instance, 2018-07-08 09:49:43.868 and 2018-07-08 09:49:43

Is it possible to parse it with the date filter plugin or is it necessary to use Grok?

2

You could transform filedate using

mutate { gsub => [ "filedate", "(....)(..)(..)", "\1.\2.\3" ] }

You should use a date filter to parse those date. For example

date { match => [ "somefield", "yyyy-MM-dd HH:mm:ss.SSS", "yyyy-MM-dd HH:mm:ss" ] }
3

Thank you!!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.