# Optimal merge policy for indexing log data

**URL:** <https://discuss.elastic.co/t/optimal-merge-policy-for-indexing-log-data/10456>\
**Category:** Elasticsearch\
**Created:** [January 22, 2013, 5:16pm UTC](https://discuss.elastic.co/t/optimal-merge-policy-for-indexing-log-data/10456 "2013-01-22T17:16:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jan\_Fiedler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jan_fiedler/32/2518_2.png) [@Jan\_Fiedler](https://discuss.elastic.co/u/Jan_Fiedler)\
**Post date:** [January 22, 2013, 5:16pm UTC](https://discuss.elastic.co/t/optimal-merge-policy-for-indexing-log-data/10456/1 "2013-01-22T17:16:03Z")

</div>

I'm working on a little project that indexes log data (using one index per  
day). For now I just have one ES node for testing doing the indexing at a  
rather low volume (up to 1 million messages / day). While monitoring the  
CPU consumption of that search node I found that it was slowly but overall  
significantly increasing over the course of the 'indexing day'. I assume  
this is due to the fact that the index size increases making certain  
operations more expensive over time. I used the hot thread api and found  
that index merge operations seem to be consuming that CPU which brings me  
to the actual question:

What would be the recommended merge policy for this special log data set.  
Its different from normal content in that it never changes (i.e. no  
updates) and never gets deleted (only a whole index gets deleted). There  
are only 'new' documents being added at a rather constant rate. I guess  
there could be special merge settings for this type of traffic.

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:55am UTC](https://discuss.elastic.co/t/optimal-merge-policy-for-indexing-log-data/10456/2 "2017-07-06T02:55:11Z")

</div>


