"best_compression" not compressing the data

Elastic Stack Elasticsearch
1

Hello team,

I was applying "best_compression" compressing type to my existing elasticsearch indices, but I don't see any significant reduction happening in the indices size. The steps followed were :

  1. Close the indices
  2. Updated the index settings with "index.codec":"best_compression"
  3. Open the indices

I see the index settings are being updated with best_compression, but on comparing the indices stats before and after updating compression type, there is no significant reduction happening in the size, It's hardly in few bytes. I had also tried forcemerge. But got nothing.

There are multiple indices which varies in size from 500mb - 1.5 gb. The elasticsearch version is 7.17. Could anyone help with this ?

Thanks !

2

Enabling best compresion just changes the compression algorithm to a more aggressive one. Data in Elasticsearch is compressed by default, so making this change will not necessarily result in huge space savings. In my experience 10-20% is common, but it will depend on your data.

As segments are immutable in Elasticsearch you need to forcemerge all segments for this to take effect. One way of ensuring that all data is compressed using the updated codec is to forcemerge the index down to a single shard. Have you tried this?

3

Thanks Christian. It is resolved.

I had to set "max_num_segments" to 1 for the merge to happen when forcemerging the indices.
Seems by default it checks if a merge necessarily needs to be executed (executes only if needed) when we don't give the "max_num_segments" parameter.

Thanks again !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.