Index size Reduced in Elasticsearch even When Number of Documents Increased

Elastic Stack Elasticsearch
1

Hi There,

I am storing incremental data (Everyday 0.7 - 1 GB of data gets added) in Elasticsearch. The interesting observation I found is that :

The index size in the elasticsearch decreased with time while the number of documents in that index increased with time.

Index stat Yesterday :
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open my_index CFnG6rtNQAGDmI4zpGrFSb 5 1 85458039 83704 31.6gb 31.6gb

Index stat Today :
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open my_index CFnG6rtNQAGDmI4zpGrFSb 5 1 86428292 83704 27.5gb 27.5gb

As you can see the number of docs in the index increased from 85458039 to 86428292 (Increase of 970253 documents) while the index size reduced from 31.6gb to 27.5gb.

  • How is that possible ?

  • Am I loosing some data or is there some compaction algorithm working behind the scenes?

  • I have specified replica as 1, so why are store.size and pri.store.size same?

  • Why is my Index health is Yellow?

Please enlighten me.

2

there is some compaction algorithm running the background. See https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-merge.html

Your index health is yellow because you are very likely only running on a single node, but have configured this index to have a replica. Replicas are never put on the same node as a primary, so having a copy of a shard in single node cluster is not possible.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.