But with this config, I am not able to achieve any index compression, rather it is bloating.That is for a log file of size less than 2 GB the index size come around 2 GB.
Kindly help me with understanding why there are no compression or what is the mistake that i have done the in the above setup.
@Christian_Dahlqvist: Does this have any problem that would hamper the compression? Or is there any other settings that i will have to tweak to achieve compression
I see that you seem to be using the default dynamic mappings. These do index every field other as text and keyword, which adds a lot of flexibility but can also take up quite a bit of extra space on disk. I would recommend you go through your mappings and optimize them according to these guidelines.
The best_compression codec applies to the JSON source, so these mappings will not be affected.
@Christian_Dahlqvist: thank you. I will try to follow the guidelines to optimise the disk usage.
Do you have any suggestion or link which says exactly how to correctly set the codec ? will setting the codec via template make sure it is applied?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.