# Is compression done globally or on each document **URL:** \ **Category:** Elasticsearch\ **Created:** [February 19, 2012, 3:14pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747 "2012-02-19T15:14:44Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [February 19, 2012, 3:14pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/1 "2012-02-19T15:14:44Z")
Hi , I would like to know if the compression option in \_source is done over all the documents in a type or on individual documents. Thanks Vineeth ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 20, 2012, 1:08pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/2 "2012-02-20T13:08:45Z")
\_source compression is done per document. On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > Hi , > > I would like to know if the compression option in \_source is done over all the documents in a type or on individual documents. > > Thanks > Vineeth ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [February 20, 2012, 1:17pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/3 "2012-02-20T13:17:10Z")
It would have helped a lot if it was done globally for a index. Thanks Vineeth On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > \_source compression is done per document. > > On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > > Hi , > > I would like to know if the compression option in \_source is done over all > the documents in a type or on individual documents. > > Thanks > Vineeth ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 20, 2012, 1:30pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/4 "2012-02-20T13:30:49Z")
On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote: > It would have helped a lot if it was done globally for a index. That would mean having to recompress all of your data every time you index a document. If your index is basically read-only, then after indexing all your data, you could use the optimize API clint > Thanks > Vineeth > > On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > \_source compression is done per document. > > ``` > On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > > > Hi , > > > > I would like to know if the compression option in _source is > > done over all the documents in a type or on individual > > documents. > > > > Thanks > > Vineeth > > > > > > ``` ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [February 20, 2012, 1:56pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/5 "2012-02-20T13:56:48Z")
AFAIK one does not need to un-compress the whole zip file just to get a single file. We can also retrieve one file from the whole compression at a time. Thanks Vineeth On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote: > > > It would have helped a lot if it was done globally for a index. > > That would mean having to recompress all of your data every time you > index a document. > > If your index is basically read-only, then after indexing all your data, > you could use the optimize API > > clint > > > Thanks > > Vineeth > > > > On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > \_source compression is done per document. > > > > ``` > > On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > > > > > Hi , > > > > > > I would like to know if the compression option in _source is > > > done over all the documents in a type or on individual > > > documents. > > > > > > Thanks > > > Vineeth > > > > > > > > > > ``` ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 20, 2012, 2:10pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/6 "2012-02-20T14:10:14Z")
It more than that, we can actually block compress data, but it still has a cost and its not implemented. On Monday, February 20, 2012 at 3:56 PM, Vineeth Mohan wrote: > AFAIK one does not need to un-compress the whole zip file just to get a single file. > We can also retrieve one file from the whole compression at a time. > > Thanks > Vineeth > > On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley \<[clint@traveljury.com](mailto:clint@traveljury.com) ([mailto:clint@traveljury.com](mailto:clint@traveljury.com))\> wrote: > > > On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote: > > > > > It would have helped a lot if it was done globally for a index. > > > > That would mean having to recompress all of your data every time you > > index a document. > > > > If your index is basically read-only, then after indexing all your data, > > you could use the optimize API > > > > clint > > > > > Thanks > > > Vineeth > > > > > > On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) ([mailto:kimchy@gmail.com](mailto:kimchy@gmail.com))\> wrote: > > > \_source compression is done per document. > > > > > > ``` > > > On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > > > > > > > Hi , > > > > > > > > I would like to know if the compression option in _source is > > > > done over all the documents in a type or on individual > > > > documents. > > > > > > > > Thanks > > > > Vineeth > > > > > > > > > > > > > > ``` ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [February 20, 2012, 3:16pm UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/7 "2012-02-20T15:16:01Z")
I feel there would be people who are willing to accept that cost as compression over text works like magic.( i have seen 1:6 size with compressed data). I feel its a good feature , can i go ahead and file a issue ? Thanks Vineeth On Mon, Feb 20, 2012 at 7:40 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > It more than that, we can actually block compress data, but it still has > a cost and its not implemented. > > On Monday, February 20, 2012 at 3:56 PM, Vineeth Mohan wrote: > > AFAIK one does not need to un-compress the whole zip file just to get a > single file. > We can also retrieve one file from the whole compression at a time. > > Thanks > Vineeth > > On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote: > > > It would have helped a lot if it was done globally for a index. > > That would mean having to recompress all of your data every time you > index a document. > > If your index is basically read-only, then after indexing all your data, > you could use the optimize API > > clint > > > Thanks > > Vineeth > > > > On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > \_source compression is done per document. > > > > ``` > > On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote: > > > > > Hi , > > > > > > I would like to know if the compression option in _source is > > > done over all the documents in a type or on individual > > > documents. > > > > > > Thanks > > > Vineeth > > > > > > > > > > ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:38am UTC](https://discuss.elastic.co/t/is-compression-done-globally-or-on-each-document/6747/8 "2017-07-06T03:38:46Z")
Closed on July 6, 2017, 3:38am UTC