Kibana index patterns and date math

[mrkita (Morgan Kita)]

Maybe it's already possible, but it doesn't seem so to me.

I want to create an index pattern in Kibana that slices my elastic search data to the last X number of days (something like the last 3). I know that I can create aliases in elastic search, but as far as I can tell they are static at the time of creation and don't directly support "date math" (as defined here: https://www.elastic.co/guide/en/elasticsearch/reference/current/date-math-index-names.html).

As far as I can tell the kibana "create pattern index" tool does not allow anything other than a wild card character (no date math), which means my indices (which contain a day based time stamp) could at most be limited to a specific month, and even then I would have to create a new pattern for each month, and then update all my views and visualizations to reference the new data. Plus a month is way more data than I need.

Is there another option here? Or is that a potential feature request? It does make a big difference in performance on resource limited machines (see my other post about the raspberry pi).

[Christian_Dahlqvist]

Could you not add an alias with a filtered query to your template like this:

PUT _template/test
{
  "index_patterns" : ["test-*"],
  "aliases" : {
    "last_3_days" : {
      "filter" : {
        "range": { "@timestamp": { "gte": "now-3d" } }
      }
    }
  }
}

[mrkita (Morgan Kita)]

Ok I didn't understand the "template" concept at the time I first read this. Thank you for the suggestion, and that is in the right direction, but if I understand index templates and aliases correctly, then this will not automatically remove indices that have been added previously but are now outside this 3 days window? In other words this only takes care of autoadding new indexed date to the alias, but cleanup must be done manually?

EDIT:
After writing my last edit... by random google chance I learned about "Elasticsearch Curator". After doing my own integration with my docker environment and tweaking it a bit, I was easily able to get exactly this behavior with the curator. Why is this thing not a standard part of the officially support ELK-Docker-Stack, it's exceptionally helpful!