Time Based Index Query Performance

Elastic Stack Kibana
1

I was reading through this old thread: Kibana search performance over time-based indices.

Based on the dialog in that thread, it seems that the latest versions of ES/Kibana should automatically limit searches in the discover tab to just the indices that contain the time range being queried for. I am not sure if I am seeing that behavior though.

I have logstash setup with a typical pattern to index by day and I currently have 4 days of indices with an index pattern of "logstash*". On the discover tab, I am querying for the last hour of logs.

I clicked the "inspect" menu item and copied the request. I then pasted the request into the developer console for a request to POST /logstash*/_search and added "profile":"true" to the request to get profile information.

The profiling results that come back are indicating that all of my indicies are being searched, not just the current days. Search performance is similar between the discover tab and the console request as well.

Am I doing something wrong? Is there something else going on under the hood in the discover tab like its making a different kind of search request? My index pattern does have the timestamp field specified as well. I am running version 7.4 on AWS Elasticsearch service.

2

Upon further inspection of the profiling data, I see that all the queries to the older indices are of type "MatchNoDocsQuery". Is this roughly equivalent to a no-op? All of those are super fast, <10 microseconds. So, am I essentially seeing the expected behavior in that, while the query is being sent to every index, ES is intelligently rewriting them to MatchNoDocsQuery so that they essentially do nothing and return immediately?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.