Store.size decreased drastically after reindexing data from 5.6 ES cluster to 7.6 ES cluster

[Bharath_Italapur (Bharath reddy italapur)]

Hi,

Thank you in advance,

I reindexed 5 Gb data from 5.6 ES cluster to 7.6 ES cluster. The data size decreased to 2.7 Gb, But the document count is the same. Can I know why this is happening?

The only difference between 5.6 ES cluster and 7.6 ES cluster is mapping, they are as follows.

1. There are no types in 7.6 ES cluster. (We are having single type mapping in 5.6 ES cluster so I removed type name in the mapping )

2. For the time zone, I am using XXX in 7.6 ES cluster instead of Z in 5.6 ES cluster

3. Instead of integer and long in 5.6 ES cluster, we are using double in 7.6 es cluster.

ES-5.6

ES- 7.6

[dadoonet (David Pilato)]

One other "tiny" difference is that there are 2 major versions away and same happens for Lucene which is used behind the scene.

Lot of things changed in the meantime. For example, geo points are taking less space on disk.

[Bharath_Italapur (Bharath reddy italapur)]

Thank you for the quick response.

[quote="dadoonet, post:2, topic:223400, full:true"]
One other "tiny" difference
[/quote]

So this won't affect the data right? I just want to make sure.

[Christian_Dahlqvist]

It may also be that they are at different stages of merging. If you forcemerge down to a single segment you can compare size more reliably.

[Bharath_Italapur (Bharath reddy italapur)]

Hi Christian,

Thank you for the response.

@Christian_Dahlqvist

If you forcemerge down to a single segment you can compare size more reliably.

I am not familiar to this process. Can you please let me know how to do this.

[Christian_Dahlqvist]

Have a look at the dics.

[Bharath_Italapur (Bharath reddy italapur)]

Hi Christian,

You want me to forcemerge the source index to know the actual size of the index. It is a production cluster. Just want to make sure that it wont effect the data right?

[Christian_Dahlqvist]

It wont affect the data but can be quite I/O intensive.