Space consumption after reindex

Elastic Stack Elasticsearch
1

Hi folks

we are trying to migrate from 5.5.1 to 7.10.2. After upgrade to 6.8 I'm trying to reindex indices which were created on 5.5.1.

curl -sq -X POST "http://elasticsearch:9200/_reindex?pretty" -H 'Content-Type: application/json' -d"
  {
    \"source\": {
      \"index\": \"filebeat-2021.03.10\"
    },
    \"dest\": {
      \"index\": \"reindexed-filebeat-2021.03.10\"
    }
  }"

After reindex I create an alias and remove the source index:

  curl -sq -X POST "http://elasticsearch:9200/_aliases?pretty" -H 'Content-Type: application/json' -d"
  {
    \"actions\": [{
      \"add\": {
        \"index\": \"reindexed-filebeat-2021.03.10\",
        \"alias\": \"filebeat-2021.03.10\"
      }
    },
    {
      \"remove_index\": {
        \"index\": \"filebeat-2021.03.10\"
      }
    }]
  }"

When I started the server had 66% free hdd space. After reindex the half of all source indices, I have only 15% free hdd space. Desprite the new index consumes mostly less space than the source one. Where the space goes?

I've tried already force merge:

curl -XPOST http://elasticsearch:9200/_forcemerge

and with only_expunge_deletes:

curl -sq -XPOST http://elasticsearch:9200/_forcemerge?only_expunge_deletes=true

Nothing helps. Does anyone has an idea?

Best regards
speechkey

2

What's the output from _cat/nodes?v?

3

Given that you have changed the index prefix I would suspect the Filebeat index templates might not apply which could mean you are using default mappings, which can take up more space. Compare the mappings for an old index with a reindexed one and see if there are any significant differences.

4

Thanks for replies. The problem disappears over night.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.