Problem with realtime GET

Elastic Stack Elasticsearch
1

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a
project where I need to guarantee that once a document is index, all
subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a
document and then immediately reads it. The test works ok on a single
node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The
failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
https://gist.github.com/1221648. That class is all you need to run the
test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel

2

Hey, thanks for the recreation. Managed to recreate it as well here, opened
an issue and fixed it in 0.17 and master branch:
Realtime Get: Under high concurrent indexing and immediate get, a get might be missed while flushing · Issue #1344 · elastic/elasticsearch · GitHub.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev pavel.penchev@gmail.comwrote:

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a
project where I need to guarantee that once a document is index, all
subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a
document and then immediately reads it. The test works ok on a single node
setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The failed
documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
ES problem with realtime get · GitHub. That class is all you need to run the
test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel

3

Superb!

Many thanks,
Pavel

On 18.09.2011 13:45, Shay Banon wrote:

Hey, thanks for the recreation. Managed to recreate it as well here,
opened an issue and fixed it in 0.17 and master branch:
Realtime Get: Under high concurrent indexing and immediate get, a get might be missed while flushing · Issue #1344 · elastic/elasticsearch · GitHub.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev
<pavel.penchev@gmail.com mailto:pavel.penchev@gmail.com> wrote:

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working
on a project where I need to guarantee that once a document is
index, all subsequent reads will find it.

I've created a small test that runs 50 threads and each thread
writes a document and then immediately reads it. The test works ok
on a single node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions).
The failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
https://gist.github.com/1221648. That class is all you need to run
the test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel