# Index sorting would cause indexed data movement?

**URL:** <https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978>\
**Category:** Elasticsearch\
**Created:** [May 22, 2024, 9:34am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978 "2024-05-22T09:34:25Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![chrwhy](https://avatars.discourse-cdn.com/v4/letter/c/f17d59/32.png) [@chrwhy](https://discuss.elastic.co/u/chrwhy)\
**Post date:** [May 22, 2024, 9:34am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/1 "2024-05-22T09:34:25Z")

</div>

Anybody know that index-sorting data is physically or logically sorted on disk?

here is my question: if there are existing sorted data, when inserting a new record into the index, how will the new index-sorting data be organized? physically re-sort or logically re-sort ? and why?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 22, 2024, 10:14am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/2 "2024-05-22T10:14:47Z")

</div>

Welcome!

It's done as soon as we write a new segment. Which means for all the data in this segment.  
When we are merging segments, it's done again.

---

<div class="post-metadata">

**Author:** ![chrwhy](https://avatars.discourse-cdn.com/v4/letter/c/f17d59/32.png) [@chrwhy](https://discuss.elastic.co/u/chrwhy)\
**Post date:** [May 22, 2024, 10:19am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/3 "2024-05-22T10:19:36Z")

</div>

but, what about the new segments which are still being appending new data to?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 22, 2024, 10:23am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/4 "2024-05-22T10:23:45Z")

</div>

Segments are immutable so not changed or appended to once written.

---

<div class="post-metadata">

**Author:** ![chrwhy](https://avatars.discourse-cdn.com/v4/letter/c/f17d59/32.png) [@chrwhy](https://discuss.elastic.co/u/chrwhy)\
**Post date:** [May 22, 2024, 11:01am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/5 "2024-05-22T11:01:41Z")

</div>

do you mean once the segment is refreshed, everything is settled? No in-progress state?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 22, 2024, 11:04am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/6 "2024-05-22T11:04:41Z")

</div>

New segments are created based on data in the transaction log. Once they have been created they do not change. Periodically segments are merged, which results in new larger segments being created before the old ones that were merged are deleted.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 22, 2024, 11:23am UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/7 "2024-05-22T11:23:05Z")

</div>

Some resources to understand this:

- [Merge | Elasticsearch Guide [8.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-merge.html)
- [cat segments API | Elasticsearch Guide [8.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-segments.html)

[![](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed53b28773c28932a9065603a87d103c92ebe05b.jpeg "Lucene merges while indexing all of Wikipedia (English)") ](https://www.youtube.com/watch?v=YW0bOvLp72E)

---

<div class="post-metadata">

**Author:** ![chrwhy](https://avatars.discourse-cdn.com/v4/letter/c/f17d59/32.png) [@chrwhy](https://discuss.elastic.co/u/chrwhy)\
**Post date:** [May 22, 2024, 12:02pm UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/8 "2024-05-22T12:02:19Z")

</div>

so, it is like once refresh was triggered, new segments would be created from the transaction log, the index-sorting data would be created during that process, except that when segment merging happens, the index-sorting data would be merged again too.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 22, 2024, 12:34pm UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/9 "2024-05-22T12:34:02Z")

</div>

Correct

---

<div class="post-metadata">

**Author:** ![chrwhy](https://avatars.discourse-cdn.com/v4/letter/c/f17d59/32.png) [@chrwhy](https://discuss.elastic.co/u/chrwhy)\
**Post date:** [May 22, 2024, 2:02pm UTC](https://discuss.elastic.co/t/index-sorting-would-cause-indexed-data-movement/359978/10 "2024-05-22T14:02:32Z")

</div>

yeah, well noted, thank you two so much @dadoonet @Christian_Dahlqvist
