Elasticsearch Filter - Daily index search

Elastic Stack Logstash
1

When using the elasticsearch filter plugin to query previous documents, I'm wondering if there is an option to specify the previous days index.

For example, I have the below filter:

filter {
  elasticsearch {
          hosts => ["localhost:9200"]
          index => "ciscoasa-%{+y}.%{+MM}.%{+dd}"
          query => "connection_id:%{[connection_id]}"
          fields => { "direction" => "direction" }
  }
}

This produces errors when the connections overlaps overnight. i.e the Built connection arrives at 23:59 and the Teardown connection (which this filter is triggered on) comes in the next day at 00:01.

I'm not wanting to specify _all or leave it empty to query all indexes as there is a lot of indexes and believe this will slow down the query.

The only way I can think is to use "ciscoasa-%{+y}.%{+MM}*" which would only cause it to produce errors when the month changes, albeit only for a few events.

Any suggestions?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.