Is compression done globally or on each document

Elastic Stack Elasticsearch
1

Hi ,

I would like to know if the compression option in _source is done over all
the documents in a type or on individual documents.

Thanks
Vineeth

2

_source compression is done per document.

On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:

Hi ,

I would like to know if the compression option in _source is done over all the documents in a type or on individual documents.

Thanks
Vineeth

3

It would have helped a lot if it was done globally for a index.

Thanks
Vineeth

On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon kimchy@gmail.com wrote:

_source compression is done per document.

On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:

Hi ,

I would like to know if the compression option in _source is done over all
the documents in a type or on individual documents.

Thanks
Vineeth

4

On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote:

It would have helped a lot if it was done globally for a index.

That would mean having to recompress all of your data every time you
index a document.

If your index is basically read-only, then after indexing all your data,
you could use the optimize API

clint

Thanks
Vineeth

On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon kimchy@gmail.com wrote:
_source compression is done per document.

    On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:
    
    > Hi ,
    > 
    > I would like to know if the compression option in _source is
    > done over all the documents in a type or on individual
    > documents.
    > 
    > Thanks
    >           Vineeth
    > 
    >
5

AFAIK one does not need to un-compress the whole zip file just to get a
single file.
We can also retrieve one file from the whole compression at a time.

Thanks
Vineeth

On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley clint@traveljury.comwrote:

On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote:

It would have helped a lot if it was done globally for a index.

That would mean having to recompress all of your data every time you
index a document.

If your index is basically read-only, then after indexing all your data,
you could use the optimize API

clint

Thanks
Vineeth

On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon kimchy@gmail.com wrote:
_source compression is done per document.

    On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:

    > Hi ,
    >
    > I would like to know if the compression option in _source is
    > done over all the documents in a type or on individual
    > documents.
    >
    > Thanks
    >           Vineeth
    >
    >
6

It more than that, we can actually block compress data, but it still has a cost and its not implemented.

On Monday, February 20, 2012 at 3:56 PM, Vineeth Mohan wrote:

AFAIK one does not need to un-compress the whole zip file just to get a single file.
We can also retrieve one file from the whole compression at a time.

Thanks
Vineeth

On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley <clint@traveljury.com (mailto:clint@traveljury.com)> wrote:

On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote:

It would have helped a lot if it was done globally for a index.

That would mean having to recompress all of your data every time you
index a document.

If your index is basically read-only, then after indexing all your data,
you could use the optimize API

clint

Thanks
Vineeth

On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon <kimchy@gmail.com (mailto:kimchy@gmail.com)> wrote:
_source compression is done per document.

    On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:

    > Hi ,
    >
    > I would like to know if the compression option in _source is
    > done over all the documents in a type or on individual
    > documents.
    >
    > Thanks
    >           Vineeth
    >
    >
7

I feel there would be people who are willing to accept that cost as
compression over text works like magic.( i have seen 1:6 size with
compressed data).

I feel its a good feature , can i go ahead and file a issue ?

Thanks
Vineeth

On Mon, Feb 20, 2012 at 7:40 PM, Shay Banon kimchy@gmail.com wrote:

It more than that, we can actually block compress data, but it still has
a cost and its not implemented.

On Monday, February 20, 2012 at 3:56 PM, Vineeth Mohan wrote:

AFAIK one does not need to un-compress the whole zip file just to get a
single file.
We can also retrieve one file from the whole compression at a time.

Thanks
Vineeth

On Mon, Feb 20, 2012 at 7:00 PM, Clinton Gormley clint@traveljury.comwrote:

On Mon, 2012-02-20 at 18:47 +0530, Vineeth Mohan wrote:

It would have helped a lot if it was done globally for a index.

That would mean having to recompress all of your data every time you
index a document.

If your index is basically read-only, then after indexing all your data,
you could use the optimize API

clint

Thanks
Vineeth

On Mon, Feb 20, 2012 at 6:38 PM, Shay Banon kimchy@gmail.com wrote:
_source compression is done per document.

    On Sunday, February 19, 2012 at 5:14 PM, Vineeth Mohan wrote:

    > Hi ,
    >
    > I would like to know if the compression option in _source is
    > done over all the documents in a type or on individual
    > documents.
    >
    > Thanks
    >           Vineeth
    >
    >