Lots of segments per index

Elastic Stack Elasticsearch
1

Hi All,

We have monthly indexes (currently 11 months) with 22 shards per index. I
am seeing what seems to be a lot of segments per index (roughly 1200 to
1380 segments). The older indexes should have very little, if any updates
occurring on them. From everything I have read it sounds like ES should be
automatically merging segments, but now I am a bit concerned that this is
not occurring. I know we can manually run an optimize, but will need to
allocate another resource to do that work (as to not impact current
system). I am fairly new to ES (if that isn't obvious) and am really
trying to understand we have an issue or not.

Rough index stats:

11 indexes
22 shards per index
65 million docs per index
350 GB per index

Any info, suggestions, etc. are greatly appreciated.

Thanks!

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c6bedad4-9c24-4e92-94ef-c4fc0e51a7ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

It merges segments in response to indexes and updates so an index that
doesn't change will not have merges. You can manually optimize the index
once, when it is mostly done with updates. Once the index is optimized
further calls to optimize with the same parameters are noops.

You can't really ads more resources to optimize though, its something done
on each shard.
On May 20, 2015 7:23 PM, mike.giardinelli@gmail.com wrote:

Hi All,

We have monthly indexes (currently 11 months) with 22 shards per index. I
am seeing what seems to be a lot of segments per index (roughly 1200 to
1380 segments). The older indexes should have very little, if any updates
occurring on them. From everything I have read it sounds like ES should be
automatically merging segments, but now I am a bit concerned that this is
not occurring. I know we can manually run an optimize, but will need to
allocate another resource to do that work (as to not impact current
system). I am fairly new to ES (if that isn't obvious) and am really
trying to understand we have an issue or not.

Rough index stats:

11 indexes
22 shards per index
65 million docs per index
350 GB per index

Any info, suggestions, etc. are greatly appreciated.

Thanks!

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c6bedad4-9c24-4e92-94ef-c4fc0e51a7ed%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c6bedad4-9c24-4e92-94ef-c4fc0e51a7ed%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0QO%3DQ2T4c6-tZQH6x7YjW3eLZEeF1K_Zz3XSP-dTK7HQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.