Messages getting lost during insertion

Elastic Stack Elasticsearch
1

Hi All,
I am writing messages into elasticsearch using a single thread, and
using the call:
transportClient.prepareIndex(index, type, id)
.setSource(node)
.execute()
.actionGet();
If I write messages continuously, I observe that some messages are
lost somewhere in transit, and my query returns less entries than I
write.

If I use elasticsearchhead UI, I see the status of the index as
test
size: 68.9mb (68.9mb)
docs: 42795 (42820)

My index status returns:
docs: {
num_docs: 42795
max_doc: 42820
deleted_docs: 25
}

The max_docs is less than the number of entries that I pushed into
ElasticSearch. And why are some messages marked as deleted?
I am using ElasticSearch 0.16.2.

Regards,
Sameer

2

Are you sure you don't index the same document several time (same id)? If
you want to be sure, you can call setOpType(IndexRequest.OpType.CREATE) just
after setSource, and indexing will fail if there is already an existing doc
with the same id.

On Tue, Sep 13, 2011 at 12:50 PM, Sameer sameerpokarna@gmail.com wrote:

Hi All,
I am writing messages into elasticsearch using a single thread, and
using the call:
transportClient.prepareIndex(index, type, id)
.setSource(node)
.execute()
.actionGet();
If I write messages continuously, I observe that some messages are
lost somewhere in transit, and my query returns less entries than I
write.

If I use elasticsearchhead UI, I see the status of the index as
test
size: 68.9mb (68.9mb)
docs: 42795 (42820)

My index status returns:
docs: {
num_docs: 42795
max_doc: 42820
deleted_docs: 25
}

The max_docs is less than the number of entries that I pushed into
Elasticsearch. And why are some messages marked as deleted?
I am using Elasticsearch 0.16.2.

Regards,
Sameer