Disabling Elasticsearch Index compression version 7.10+

Elastic Stack Elasticsearch
1

Hi, I am new to Elasticsearch and I have a question around Elasticsearch compression. I know it uses the codec default and best_compression when creating an index but since my backend storage already provides LZ4 compression, is there away to disable compression at the Elasticsearch Index level? This way I can free up CPU cycles on the ES nodes and let the storage array does all the compression for me.
Thanks in advance.

2

I don't believe you can disable it, the only options are LZ4 or DEFLATE.

3

Thanks, Kathleen. So if someone who doesn't use DAS for storage, then they're stuck with the default settings? I saw some article talking about writing your own codec to disable compression but that requires recompiling ES code base? Is that an accurate statement?

4

Hi @tommyd, Welcome to the community!

Yes no way AFAIK to disable compression, it is a core part of Elastic.

Unless you are at an extreme scale, Lucene is really, really efficient, it seems like a lot of work and risk and maintenance for perhaps a small gain... especially if you are just getting started with Elastic.

It seems like spending your time on good mappings, index management queries, etc., might be a better use of your time.

Just one opinion.

1 Like
5

Thanks Stephen. Appreciated for your help and Kathleen's help!

Cheers!

6

For completeness, yes, you would need to compile and repackage Elasticsearch (and quite possibly some of its dependencies) to disable compression.

I wouldn't be surprised to see worse performance after doing so too - the OS page cache would be working with the uncompressed data and so will fill up much much faster, and this is an extremely important factor for performance. There's a good reason this isn't offered as an option.

1 Like
7

Thanks, David! This helps a lot.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.