# Compressing and forcemerging past time-based indices **URL:** \ **Category:** Elasticsearch\ **Created:** [November 20, 2020, 4:39am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060 "2020-11-20T04:39:12Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 4:39am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/1 "2020-11-20T04:39:13Z")
I have a bunch of past `monthly` indices that are `read-only` since no data is being indexed to them. I was thinking of applying `best_compression` on them and then invoke the `_forcemerge` api to reduce the `num_segments` to `1`. I can `close` those monthly indices --\> apply `best_compression` --\> `open` them again --\> `forcemerge` them. Is this the correct approach? Or that best\_compression `cannot` be applied to existing indices? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 20, 2020, 4:49am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/2 "2020-11-20T04:49:59Z")
> [@sandeepkanabar](#): > > I can `close` those monthly indices --\> apply `best_compression` --\> `open` them again --\> `forcemerge` them. Yep! ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 5:31am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/3 "2020-11-20T05:31:23Z")
Thank you very much đź‘Ť Will it have any performance impact on queries? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 20, 2020, 5:32am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/4 "2020-11-20T05:32:49Z")
Better compression isn't free. It'll have some impact, but how much you will need to measure. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 5:34am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/5 "2020-11-20T05:34:58Z")
Understood. Thanks. I was hoping that with forcemerging to 1 segment, hopefully it may not have `that much impact`. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 20, 2020, 5:35am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/6 "2020-11-20T05:35:36Z")
It shouldn't, but it's not something I think you will find anyone will guarantee. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 5:37am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/7 "2020-11-20T05:37:05Z")
Thank you. Yes agree on that. Will measure it. One question here: In case I opt to `best_compression` and find that the query times have `increased`, what option do I have? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 20, 2020, 5:39am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/8 "2020-11-20T05:39:07Z")
You should be able to revert that with a settings change and another force merge. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 5:39am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/9 "2020-11-20T05:39:47Z")
Awesome. Thank you very much. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 20, 2020, 6:08am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/10 "2020-11-20T06:08:05Z")
Using best\_compression primarily adds overhead at indexing time so should not affect query latencies at all. I also do not think you need to close the index. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 6:09am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/11 "2020-11-20T06:09:37Z")
Wow. That's great to hear. So just set the compression and forcemerge to 1 segment? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 20, 2020, 6:10am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/12 "2020-11-20T06:10:24Z")
Yes. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 6:10am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/13 "2020-11-20T06:10:54Z")
Excellent. Thanks. By the way, if we have forcemerged before to 5 segments, we can again forcemerge to 1 segment. Correct? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 20, 2020, 6:11am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/14 "2020-11-20T06:11:43Z")
No, as far as I know that will not change anything’s my. Be aware that forcemerging large shards can take a long time and result in a good amount of disk I/O. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 6:20am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/15 "2020-11-20T06:20:42Z")
Noted. Thanks for all your inputs. Much appreciated. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 20, 2020, 6:59am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/16 "2020-11-20T06:59:57Z")
@Christian_Dahlqvist and @warkolm - truly amazed at the support you guys are providing. Very much grateful. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 25, 2020, 4:11am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/17 "2020-11-25T04:11:13Z")
> [@Christian\_Dahlqvist](#): > > forcemerging large shards can take a long time and result in a good amount of Fisk I/O Hi @Christian_Dahlqvist - would it be okay if I forcemerge multiple indices (say 3 to 5 indices) in `parallel` in a huge cluster of 50 TB (10 data nodes and 3 master nodes. Each data node is 8 TB with 16 cores and 56 GB RAM) or that the recommendation is to forcemerge only `1 index` at a time? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 25, 2020, 7:21am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/18 "2020-11-25T07:21:30Z")
It typically depends more on how fast your storage is than the amount of RAM or CPU as it results in a lot of disk I/O and used up disk space while in progress. Try and see how if affects your cluster and what limit you can tolerate. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 25, 2020, 7:44am UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/19 "2020-11-25T07:44:35Z")
Note that there is only one force-merge thread. If a force merge is already running and you request another then the later requests are placed in a queue and processed in turn. ---
**Author:** ![sandeepkanabar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeepkanabar/32/79399_2.png) [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Post date:** [November 25, 2020, 1:52pm UTC](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060/20 "2020-11-25T13:52:41Z")
Thanks Christian. The data nodes SSDs have max IOPS of 5000 with Max throughput being 200 MB/sec. [Next page](https://discuss.elastic.co/t/compressing-and-forcemerging-past-time-based-indices/256060.md?page=2)