Hi,
I want to use ILM to create a new index per day or month.
my question is how it going to affect search performance if old indexes will move to the warm | cold stage with pure HW.
let's say that I have a query range last day or last week. according to ILM documentation, the alias point to all historical indexes, which means Elastic will try to search the documents from all indexes - even old indexes stored on pure HW - doesn't it make the search slow?
Welcome to our community! 
I guess that depends on your hardware at a first level.
Hi,
Is there any optimization that Elastic know do search only on relevant indexes instead of search on very old indexes ( in case im looking on last day range)
Tnx
Have a look at this thread.
In older versions this was indeed an issue and Kibana has over the years implemented a few different workarounds. In recent versions this is however no longer considered an issue and all indices belonging to the pattern are queried.
My question is different i think..
what is the improvements between 2 scenarios in search operation for last month data:
option 1: one big index
option 2: monthly indexes - that most of them moved to slowly machines (warm | cold)
is there any optimizations that ELK know not to go to old indexes since im looking to new data even if im using index alias that point to all indexes?
The most efficient way to delete data in Elasticsearch is to delete complete indices. This is why time-based indices are generally recommended when dealing with data having a specific retention period. Deleteing data from a single index using delete-by-query is a LOT less efficient and adds overhead.
Elasticsearch will query all indices but each node will efficiently be able to tell if a shard holds data that could be relevant, e.g. based on the timestamp field and range clauses on this field. If a shard clearly does not hold any relevant data it can quickly be skipped.
many thanks
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.