Max_doc, deleted_docs and size number jump up and down

Elastic Stack Elasticsearch
1

Hi,

We are having a process feeding the logs into our ES server (single
node, 5 shards, vanilla setup). One thing puzzles us is that the
max_doc, deleted_docs and size number are jumping up and down in "ES
head" (by >1000 documents).

The num_docs appears OK though, and we are pretty sure no delete
operation is there.

Is this expected? What is the reason?

Thanks and regards,

-- Liyu

2

If you are reindexing the same doc, it ends up deleting the previous doc and
indexing the new doc. Also, as internal merges happen within the index to
maintain it, those number may change as well.

On Thu, Oct 27, 2011 at 9:05 PM, Liyu liyuyi@gmail.com wrote:

Hi,

We are having a process feeding the logs into our ES server (single
node, 5 shards, vanilla setup). One thing puzzles us is that the
max_doc, deleted_docs and size number are jumping up and down in "ES
head" (by >1000 documents).

The num_docs appears OK though, and we are pretty sure no delete
operation is there.

Is this expected? What is the reason?

Thanks and regards,

-- Liyu

3

I had noticed this in my setup and problem turned out that I was inserting records with duplicate ids.
I was not noticing numbers jump up and down though, and total of numdocs and deleted docs was equal to the number of docs that i had inserted...

-----Original message-----
From: Liyu
Sent: 28/10/2011, 12:35 am
To: elasticsearch
Subject: max_doc, deleted_docs and size number jump up and down

Hi,

We are having a process feeding the logs into our ES server (single
node, 5 shards, vanilla setup). One thing puzzles us is that the
max_doc, deleted_docs and size number are jumping up and down in "ES
head" (by >1000 documents).

The num_docs appears OK though, and we are pretty sure no delete
operation is there.

Is this expected? What is the reason?

Thanks and regards,

-- Liyu