Unbelievable performance improvement with new Index Sorting feature

[Kaspars_Sprogis]

Came here to say huge "thank you" for all the devs who helped delivering this great improvement - Index Sorting (6.x): https://www.elastic.co/guide/en/elasticsearch/reference/6.x/index-modules-index-sorting.html

When I first used this feature I couldn't believe my eyes, how huge performance gain was especially when using together track_total_hits: false. Since I'm listing all documents always by date and for public list I don't care total count, gain was impressive, here are few charts. Just sharing in case someone finds useful.

[warkolm (Mark Walkom)]

Thanks for sharing this with us, we really appreciate it!

[deepybee (Dan Palmer-Bancel)]

That's really good to see, and thanks for the feedback as Mark mentioned!

[af_andi]

Unfortunately it's not possible to use the index sorting at an index with nested fields. Is there a chance that it will be possible somewhere in the future?

Such an performance improvement would be great but we have nested fields in all of our indices (at least because of the geoip data)

[gavenkoa (Oleksandr Gavenko)]

I wonder if sorting based on date field will improve Kibana performance too.

It takes 5-20 sec to load 100 events in "View surrounding documents" which is boring ))

[gavenkoa (Oleksandr Gavenko)]

Do I understand this feature correctly?

Previously we had named index patterns and that practice was deprecated as index can be eliminated easily by comparing with pre-calculated min/max date from index.

Now ES can eliminate segments within shards if value in query out of min/max values of sorting? And ES narrows search if it is inside min/max bound?