TSDS Best Compression On ILM Rollover?

Hey Ben,

The index.codec index setting control compression for stored fields in Lucene.
In Elasticseach we use stored fields for the _source and the _id and allows us to quickly look up values for documents that have matched (in the get api and in the fetch phase as part the search api).

However with the TSDB the stored fields for _id and _source get trimmed away when no longer needed internally to lower disk space usage. So the extra effort that would be spend on doing best compression would then be lost. With this in mind, we advice not to use index.code=best_compression, or use the this setting at all for tsdb data streams.

Martijn

2 Likes