Regarding Index Sorting

[dawiro]

Hi,
When considering index sorting would it be correct to assume the following cost/benefits:

Pros:

• faster searching for standard queries
• Better index compression

Cons:

• Significantly reduced indexing throughput, sort parameters depending.
• In general, better suited to read heavy workloads

Also, index sorting was introduced in 6.0. Has the efficiency of index sorting moved on significantly since then?

Thx
D

[spinscale (Alexander Reelsen)]

The core idea of index sorting has not changed since 6.0 - your pro's and cons are correct. One more thing, also merging can take longer as it needs to sort data while merging several segments together (even though the data in the segments is presorted).

Not sure what you refer to with faster sorting for standard queries. If that means that your regular queries involve the sorted field in the sorting, then: yes.

Is there any question hidden behind your last one?

[dawiro]

Hi @spinscale,
Yes, I'm pondering whether to turn on index sorting in one of our secondary clusters. Sort on most common fields - maybe a couple plus @timestamp. My hope is that impact on throughput will be minimal while compressing index footprint and making normal kibana useage snappier.

This is in aws on i3.2xlarge instances. The use case is logging...

[Christian_Dahlqvist]

Logging use cases tend to be indexing heavy which is exactly the opposite compared to when index sorting is recommended.

If you are having query performance problems please share more details about the cluster and the use case. Output from the cluster stats api is also useful.