Problem with realtime GET


(Pavel Penchev) #1

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a
project where I need to guarantee that once a document is index, all
subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a
document and then immediately reads it. The test works ok on a single
node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The
failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
https://gist.github.com/1221648. That class is all you need to run the
test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel


(Shay Banon) #2

Hey, thanks for the recreation. Managed to recreate it as well here, opened
an issue and fixed it in 0.17 and master branch:
https://github.com/elasticsearch/elasticsearch/issues/1344.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev pavel.penchev@gmail.comwrote:

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working on a
project where I need to guarantee that once a document is index, all
subsequent reads will find it.

I've created a small test that runs 50 threads and each thread writes a
document and then immediately reads it. The test works ok on a single node
setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions). The failed
documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
https://gist.github.com/1221648. That class is all you need to run the
test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel


(Pavel Penchev) #3

Superb!

Many thanks,
Pavel

On 18.09.2011 13:45, Shay Banon wrote:

Hey, thanks for the recreation. Managed to recreate it as well here,
opened an issue and fixed it in 0.17 and master branch:
https://github.com/elasticsearch/elasticsearch/issues/1344.

On Fri, Sep 16, 2011 at 12:31 PM, Pavel Penchev
<pavel.penchev@gmail.com mailto:pavel.penchev@gmail.com> wrote:

Hi,

I'm facing a problem with the ES realtime GET feature. I'm working
on a project where I need to guarantee that once a document is
index, all subsequent reads will find it.

I've created a small test that runs 50 threads and each thread
writes a document and then immediately reads it. The test works ok
on a single node setup but sometimes fails to find the document
on a 2 node setup (approximately 8 times in 100 000 executions).
The failed documents are actually indexed, a manual check finds them

A gist of the main class can be found here -
https://gist.github.com/1221648. That class is all you need to run
the test, for convenience a maven project is available on demand.

Any ideas, am I missing a setting?

Thanks in advance,
Pavel

(system) #4