How many shards are hit when using day-wise indices

[sandeepkanabar]

Say, I'm using day-wise indices with 5 shards and 1 replica. In kibana dashboards, when I fire a dashboard for last 7-days which in turn fires a number of underlying visualisations in that dashboard, the queries - do they hit ALL the day-wise indices (and thereby shards) or they only hit the day-wise indices (and shards) that fall in the 7-day time range?

Here are the screenshots of the elasticsearch visualisation request and response and I believe it only hits
the indices (and shards) that fall in the 7-day time range and not ALL indices. My retention is currently 150 days and thus there would be 150 indices.

The total field tells me that 40 shards are hit which looks to be correct since for last 7 days, it would hit 8 days * 5 shards = 40.

Can someone please confirm that the kibana dashboard/visulizations queries ONLY hits the indices (and shards) that fall within the time range specified in Kibana or it hits ALL indices?

[warkolm (Mark Walkom)]

@sandeepkanabar

only hit the day-wise indices (and shards) that fall in the 7-day time range

This one.

[sandeepkanabar]

Thank you for confirming and +1 for responding this swiftly.

[Christian_Dahlqvist]

The answer is that this depends on which version of the stack you are on. In recent versions, all indices are hit, but Elasticsearch is able to very efficiently identify whether an index contains any matching data or not, so it can exclude the ones that are not needed.

[sandeepkanabar]

I'm on ES 5.x here. we haven't yet upgraded to 6.x

[sandeepkanabar]

In recent versions, all indices are hit

1. From which version?
2. Does this mean that in older version, it only hit the day-wise indices that fell in the 7-day time range

[Christian_Dahlqvist]

In early versions the app index selection was based on the date in the index name. For a large part of the 5.x series a separate query was used to determine which indices that could match. I believe this was then changed to the current behaviour late in the 5 series. I do not recall exactly when these changes occurred.