How to optimize Elasticsearch index data space

Elastic Stack Elasticsearch
4

I looked at observation 2 and noticed the following things:

  1. The template used uses Elasticsearch 2.x mappings, which have changed in 5.0.
  2. Even if the template was adjusted for ES 5.0 mappings, it will not be applied to the created index as the template would not be applicable to the index name used, only an index named elk_workshop. Retrieve the mappings for the index to see what is actually in effect.
  3. Shard and segment size can have a significant impact on the indexing overhead, as compression improves with increased data volumes. Given that you index 1000000 records, I would set this up to use a single shard and _forcemerge it down to 1 segment as this can make a big difference.
  4. Whenever I keep the @message field, I often set this to not_indexed as I rarely or never query based on it.
1 Like