# How can i reduce amount of segments **URL:** \ **Category:** Elasticsearch\ **Created:** [April 11, 2016, 1:49pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997 "2016-04-11T13:49:13Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 1:49pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/1 "2016-04-11T13:49:13Z")
i have the index of the documents that is constantly updated. The perfomance of the queries i'm running, are depending from amount segment. I can cron merging. I can set merge policy figures to minimum values like 2, but there are still too many segments and queries are still slow. ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 2:48pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/2 "2016-04-11T14:48:04Z")
How many indices/shards/segments/docs do you have? ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 3:28pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/3 "2016-04-11T15:28:31Z")
1 index 6 shards(and 1 replica) 10M docs ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 3:49pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/4 "2016-04-11T15:49:02Z")
And how many segments do you have per shard? ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 3:52pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/5 "2016-04-11T15:52:19Z")
with lowest params (segmentsPerTier = 2) it is ~ 20 segments per shard after indexing all docs. without this settings it was even more ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 4:04pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/6 "2016-04-11T16:04:55Z")
20 segments per shard should be totally fine... I don't think you can have less without killing your index/update rate. I suspect you are running costly multi-term queries like wildcard, fuzzy, range or prefix queries? ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 4:08pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/7 "2016-04-11T16:08:23Z")
i run, more like this. and with multiple segments it is 3x slower than with one. Should i cron merge with number of segment=1. also update rate is not high it is like 2 docs per sec avg, but actually updates not that frequent. so i can have lower indexing/updating speed. ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 4:16pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/8 "2016-04-11T16:16:42Z")
> [@eprst](#): > > Should i cron merge with number of segment=1 Not unless your indices are static. Otherwise every optimize call will rewrite the entire index, which would incur a lot of I/O. > [@eprst](#): > > i run, more like this. and with multiple segments it is 3x slower than with one. Hmm this is the first time I hear about this. Out of curiosity could you try to share the output of the nodes hot threads API while you have some MLT queries running in a loop? ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 4:24pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/9 "2016-04-11T16:24:40Z")
what if i set another value for number of segments of the force merge? will it rewrite entire index of just segments that will be merged? By the way it is some testing results not actual production. Regarding hot threads i need to reindex all docs to have more segments cause now i have only one segment per shard (after force merge). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 4:27pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/10 "2016-04-11T16:27:52Z")
> [@eprst](#): > > what if i set another value for number of segments of the force merge? I/O activity will be a little lighter but this might still aften rewrite significant parts of the index. ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 4:32pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/11 "2016-04-11T16:32:40Z")
and can it cause any problems? ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [April 11, 2016, 4:35pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/12 "2016-04-11T16:35:07Z")
Yes. This I/O that is required for merging will compete with search operations, and your disks will have shorter lives. ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 4:38pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/13 "2016-04-11T16:38:53Z")
but if it will speed up my queries the rest of the time, it works for me. if force\_merge will take much less time than time interval i have to run it to keep suitable amount of segments. ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 11, 2016, 4:56pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/14 "2016-04-11T16:56:20Z")
hot threads for 1 segment per shard ::: {my-es-1}{PF1j\_1AKS-apK6WHe9g7Tw}{10.240.0.15}{my-es-1/10.240.0.15:9300} Hot threads at 2016-04-11T16:55:09.817Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true: 14.0% (69.8ms out of 500ms) cpu usage by thread 'elasticsearch[es-1][search][T#25]' 6/10 snapshots sharing following 2 elements java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) 13.7% (68.7ms out of 500ms) cpu usage by thread 'elasticsearch[es-1][search][T#10]' 2/10 snapshots sharing following 34 elements sun.nio.ch.NativeThread.current(Native Method) sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46) sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:737) sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727) org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:180) org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342) org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54) org.apache.lucene.codecs.lucene50.ForUtil.readBlock(ForUtil.java:187) org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.refillDocs(Lucene50PostingsReader.java:368) org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.advance(Lucene50PostingsReader.java:452) org.apache.lucene.search.TermScorer.advance(TermScorer.java:85) org.apache.lucene.search.MinShouldMatchSumScorer.advanceTail(MinShouldMatchSumScorer.java:204) org.apache.lucene.search.MinShouldMatchSumScorer.advanceTail(MinShouldMatchSumScorer.java:214) org.apache.lucene.search.MinShouldMatchSumScorer.doNext(MinShouldMatchSumScorer.java:239) org.apache.lucene.search.MinShouldMatchSumScorer.advance(MinShouldMatchSumScorer.java:188) org.apache.lucene.search.ConjunctionDISI.doNext(ConjunctionDISI.java:118) org.apache.lucene.search.ConjunctionDISI.nextDoc(ConjunctionDISI.java:151) org.apache.lucene.search.ConjunctionScorer.nextDoc(ConjunctionScorer.java:62) org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:216) org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:169) org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:772) org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:486) org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:324) org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:106) org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:363) org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:375) org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368) org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365) org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350) org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) 7/10 snapshots sharing following 10 elements sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:737) java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:647) java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1269) org.elasticsearch.common.util.concurrent.SizeBlockingQueue.take(SizeBlockingQueue.java:161) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) 10.2% (51.2ms out of 500ms) cpu usage by thread 'elasticsearch[es-1][search][T#21]' 3/10 snapshots sharing following 2 elements java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) ---
**Author:** ![eprst](https://avatars.discourse-cdn.com/v4/letter/e/a6a055/32.png) [@eprst](https://discuss.elastic.co/u/eprst)\ **Post date:** [April 12, 2016, 1:34pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/15 "2016-04-12T13:34:48Z")
what should i see on hot\_threads? sometimes there is no thread in response at all.and some times it contains the following [http://www.wepaste.com/es-%20hot%20thread/](http://www.wepaste.com/es-%20hot%20thread/) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:59pm UTC](https://discuss.elastic.co/t/how-can-i-reduce-amount-of-segments/46997/16 "2017-07-05T22:59:59Z")