# How to implement deflate (best compression)?

**URL:** https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984
**Category:** Elasticsearch
**Created:** [November 23, 2021, 8:13pm UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984 "2021-11-23T20:13:16Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 23, 2021, 8:13pm UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/1 "2021-11-23T20:13:16Z")

</div>

Hello, kindly help me out.  
My elastic version is 6.3 how can i implement best compression for the log stored by elastic.  
Thanks

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 23, 2021, 8:42pm UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/2 "2021-11-23T20:42:13Z")

</div>

Welcome to our community! 😃  
Please upgrade, 6.3 is [EOL](https://www.elastic.co/support/eol) and no longer supported. 7.15 is latest.

Take a look at [Force merge | Elasticsearch Guide [7.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/ilm-forcemerge.html). You will need to apply the setting and then run a force merge.

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 24, 2021, 7:20am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/3 "2021-11-24T07:20:23Z")

</div>

Thanks for reply  
Sorry for the wrong version i told you about elastic.  
I am using graylog 3.0.2 and elastic 6.8.20.  
Can i implement best compression without upgrade to 7.15?  
And clearly tell me what are the various ways to apply best compression or there is only force merge can be implemented.  
Thank in advance.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 24, 2021, 8:00am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/4 "2021-11-24T08:00:26Z")

</div>

I am not sure about your version but I have done it for the 7.6 version like this :

```auto
PUT _template/my_template
{
  "index_patterns": ["*"],
  "settings": {
    "index": {
      "codec": "best_compression"
    }
  },
   "mappings" : {
     "_source": {
     "excludes": [
      ]
     }
     }
}

```

After this, all your new indices will get created with BEST compression enabled.

Note: Please try this in your lower environment first before you do it in a production cluster.  
Also, the BEST compression is recommended only for logging/monitoring use cases and should not be implemented for catalog searches without doing performance benchmarking for your use case.

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 24, 2021, 10:34am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/5 "2021-11-24T10:34:02Z")

</div>

Thanks for the reply  
But one thing i want to know how this command works what i need to do because i am unable to run it directly on terminal.  
I need to write a file or else?  
Or tell me how you applied it?  
Kindly tell me.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 24, 2021, 11:09am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/6 "2021-11-24T11:09:36Z")

</div>

don't you have kibana?

from there you can run the PUT api call directly from Dev tool.

If not then run curl command like :

`curl -XPUT "http://<your cluster ip or fqdn>:9200/_template/my_template" -H 'Content-Type: application/json' -d'{ "index_patterns": ["*"], "settings": { "index": { "codec": "best_compression" } }, "mappings" : { "_source": { "excludes": [] } }}'`

once you run the above command all your new indices will have **best** compression.

If you want to do it for the existing index then you will need to close it first and then change compression setting and then open the index.

something like this:

1. Close all indices:[http://localhost:9200/\_all/\_close](http://localhost:9200/_all/_close)'
2. Apply best\_compression to all curl -XPUT '[http://localhost:9200/\_all/\_settings](http://localhost:9200/_all/_settings)' -d '{"index.codec" : "best\_compression"}'
3. Open all indices: curl -XPOST '[http://localhost:9200/\_all/\_open](http://localhost:9200/_all/_open)'

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 5:09am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/7 "2021-11-25T05:09:45Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6edb40446afbf7dd823c746f2cf621d14db40d00.png)  
Kindly guide it shows this error.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 5:52am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/8 "2021-11-25T05:52:59Z")

</div>

Is your ES running on port 9200? probably you changed the port in your configuration file?

Can you share the cluster stats or any other command output here so that we know it's connecting to the host and port correctly?

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 6:48am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/9 "2021-11-25T06:48:09Z")

</div>

It is running on 9200.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/f/af4f18521d0ef41968e5113a49dae1b73b881afb.png)

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 7:26am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/10 "2021-11-25T07:26:33Z")

</div>

can you try this and attach the output, if this works then XPUT should also work.

curl -XGET "[http://192.168.1.211:9200/\_cluster/health](http://192.168.1.211:9200/_cluster/health)"

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 7:29am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/11 "2021-11-25T07:29:36Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/5/7/5799c96161723d85b0d6e3d5b174c9fe53451ebd.png)  
Getting this when using localhost on the place of ip.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 7:38am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/12 "2021-11-25T07:38:55Z")

</div>

its just an example , you can create template the way you want:  
try this  
curl -XPUT "[http://localhost:9200/\_template/my\_template](http://localhost:9200/_template/my_template)" -H 'Content-Type: application/json' -d'{ "index\_patterns": ["\*"], "settings": { "index": { "codec": "best\_compression" } }}'

read more here: [Create or update index template API | Elasticsearch Guide [7.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates-v1.html)

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 9:54am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/13 "2021-11-25T09:54:31Z")

</div>

Thanks a lot  
How can i verify that the best compression is implemented after running the command.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 10:23am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/14 "2021-11-25T10:23:00Z")

</div>

you can check that using :

curl -XGET "[http://localhost:9200/](http://localhost:9200/)\<your\_index\_name\>/\_settings"

output will be something like:

{"twitter1":{"settings":{"index":{"codec":" **best\_compression**","number\_of\_shards":"1","provided\_name":"twitter1","creation\_date":"1637835561774","number\_of\_replicas":"1","uuid":"VqwDV\_DlRVy3n2iYYnc\_3Q","version":{"created":"7060199"}}}}}

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 10:27am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/15 "2021-11-25T10:27:41Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b72565d99ad7e2657adcf3803fc98fe1a009747e.png)  
It is showing this.  
Is it apply compression on running index?

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 10:37am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/16 "2021-11-25T10:37:53Z")

</div>

please read this: i already explained it here

> [@How to implement deflate (best compression)?](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/6):
>
> don't you have kibana? from there you can run the PUT api call directly from Dev tool. If not then run curl command like : curl -XPUT "http://\<your cluster ip or fqdn\>:9200/\_template/my\_template" -H 'Content-Type: application/json' -d'{ "index\_patterns": ["\*"], "settings": { "index": { "codec": "best\_compression" } }, "mappings" : { "\_source": { "excludes": [] } }}' once you run the above command all your new indices will have best compression. If you w…

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 10:42am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/17 "2021-11-25T10:42:35Z")

</div>

Thanks  
How can i verify in my output "codec":" best\_compression" is not shown.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 11:04am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/18 "2021-11-25T11:04:36Z")

</div>

That means the best compression is not yet applied to your index, when did you create the graylog\_0 index?

1. Did you create a template and then create the index graylog\_0?
2. if the answer to point 1 is No then, did you close the index graylog\_0 , apply best compression and then reopen the index?

---

<div class="post-metadata">

### Author: ![Linuxuser](https://avatars.discourse-cdn.com/v4/letter/l/f475e1/32.png) [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)
#### Post date: [November 25, 2021, 11:33am UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/19 "2021-11-25T11:33:18Z")

</div>

1. I did not created template.
2. I closed index from gui and then applied compression command and reopened the index.

---

<div class="post-metadata">

### Author: ![DineshNaik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dineshnaik/32/89308_2.png) [@DineshNaik](https://discuss.elastic.co/u/DineshNaik)
#### Post date: [November 25, 2021, 12:21pm UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984/20 "2021-11-25T12:21:01Z")

</div>

Not sure how you did it, but this is what you should have done for your existing index.

1. curl -XPOST "[http://localhost:9200/graylog\_0/\_close?pretty](http://localhost:9200/graylog_0/_close?pretty)"

2. curl -XPUT '[http://localhost:9200/graylog\_0/\_settings](http://localhost:9200/graylog_0/_settings)' -H 'Content-Type: application/json' -d '{"index":{"codec":"best\_compression"}}'

3. curl -X POST "[http://localhost:9200/graylog\_0/\_open?pretty](http://localhost:9200/graylog_0/_open?pretty)"

Doing it for the template will make sure any new index which gets created using the template will by default have the best compression enabled!

command for that would be : curl -XPUT "[http://localhost:9200/\_template/my\_template](http://localhost:9200/_template/my_template)" -H 'Content-Type: application/json' -d'{ "index\_patterns": ["\*"], "settings": { "index": { "codec": "best\_compression" } }}'

Hope this helps!

[Next page](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984.md?page=2)
