# Transport.tcp.compress slowing down shard relocation

**URL:** <https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033>\
**Category:** Elasticsearch\
**Created:** [September 11, 2018, 1:49am UTC](https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033 "2018-09-11T01:49:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![djtecha](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djtecha/32/54011_2.png) [@djtecha](https://discuss.elastic.co/u/djtecha)\
**Post date:** [September 11, 2018, 1:49am UTC](https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033/1 "2018-09-11T01:49:08Z")

</div>

Looks like enabling tcp compression actually slows down the transfer rate of shards. Setting this I'm basically stuck to 13Mbs even though I've set the max\_bytes\_per\_sec to -1 This has led to it taking about 1.2h to transfer a 50GB shard when it used to take 36m. Is there any thing we can do to get around this? is the max\_bytes\_per\_sec setting not respected if we use compression? On a large cluster in AWS this helps to save a bunch of money on transfer costs, but it kind of is disappointing to see such a slow transfer rate. Important to note that the CPU isn't doing anything so I wouldn't think it was restricted by that.

---

<div class="post-metadata">

**Author:** ![ctluce](https://avatars.discourse-cdn.com/v4/letter/c/a9a28c/32.png) [@ctluce](https://discuss.elastic.co/u/ctluce)\
**Post date:** [September 21, 2018, 1:47am UTC](https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033/2 "2018-09-21T01:47:40Z")

</div>

What version of Elasticsearch are you using? This might be helpful if your running 6x [slow 6x shard recovery](https://discuss.elastic.co/t/elasticsearch-6-3-0-shard-recovery-is-slow/140940/7) Only way I was able to attain configured max\_bytes\_per\_sec was to disable compression, this was due to the change in concurrency between versions. If your running a different version then adjusting [settings](https://www.elastic.co/guide/en/elasticsearch/reference/2.4/recovery.html) may help you with shard recovery/relocation time with compression enabled, specifically `indices.recovery.concurrent_streams`

---

<div class="post-metadata">

**Author:** ![djtecha](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djtecha/32/54011_2.png) [@djtecha](https://discuss.elastic.co/u/djtecha)\
**Post date:** [October 3, 2018, 6:32pm UTC](https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033/3 "2018-10-03T18:32:19Z")

</div>

Yea, looks like compression does a double compress on recovery. There's a github issue about this. Guess I'll just disable it for now and pay the bandwidth costs ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 31, 2018, 6:32pm UTC](https://discuss.elastic.co/t/transport-tcp-compress-slowing-down-shard-relocation/148033/4 "2018-10-31T18:32:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
