Index sorting would cause indexed data movement?

Elastic Stack Elasticsearch
1

Anybody know that index-sorting data is physically or logically sorted on disk?

here is my question: if there are existing sorted data, when inserting a new record into the index, how will the new index-sorting data be organized? physically re-sort or logically re-sort ? and why?

2

Welcome!

It's done as soon as we write a new segment. Which means for all the data in this segment.
When we are merging segments, it's done again.

3

but, what about the new segments which are still being appending new data to?

4

Segments are immutable so not changed or appended to once written.

1 Like
5

do you mean once the segment is refreshed, everything is settled? No in-progress state?

6

New segments are created based on data in the transaction log. Once they have been created they do not change. Periodically segments are merged, which results in new larger segments being created before the old ones that were merged are deleted.

1 Like
7

Some resources to understand this:

2 Likes
8

so, it is like once refresh was triggered, new segments would be created from the transaction log, the index-sorting data would be created during that process, except that when segment merging happens, the index-sorting data would be merged again too.

9

Correct

1 Like
10

yeah, well noted, thank you two so much @dadoonet @Christian_Dahlqvist