Hi all,
What approaches do you use to ensure your elasticsearch indices are at a
known state when writing automated tests?
Background:
We have a high-spec machine which is shared by a number of environments for
development and testing. We have continuous integration tests which run
against that machine. Thus far, the approach has been to delete known
indices and allow the test to automatically recreate them.
In recent days, a large amount of data has been added to other, unrelated
indices.It appears that what is now happening is that deleting the index
causes the node to run the shard re-allocation process. In the meantime the
test creates a document (which should automatically create the index) but
this fails because it is delayed for too long by the shard re-allocation.
Possible approaches:
Therefore we are considering approaches where we do not delete the index
all the time. We either:
a) ensure every document in the index is deleted before running each test
b) record every document we add and delete them after the test.
Could you turn off shard reallocation or throttle it down somehow?
Of the two options you have enumerated I'd go with (a) because that makes
the test more resilient to crashes and leaves the indecies in a post-test
run state which can be useful for playing around.
If you aren't using much data you may want to spin up elasticsearch as part
of your automated test suite and connect to it.
Nik
On Tue, Aug 6, 2013 at 5:46 PM, Edward Sargisson ejsarge@gmail.com wrote:
Hi all,
What approaches do you use to ensure your elasticsearch indices are at a
known state when writing automated tests?
Background:
We have a high-spec machine which is shared by a number of environments
for development and testing. We have continuous integration tests which run
against that machine. Thus far, the approach has been to delete known
indices and allow the test to automatically recreate them.
In recent days, a large amount of data has been added to other, unrelated
indices.It appears that what is now happening is that deleting the index
causes the node to run the shard re-allocation process. In the meantime the
test creates a document (which should automatically create the index) but
this fails because it is delayed for too long by the shard re-allocation.
Possible approaches:
Therefore we are considering approaches where we do not delete the index
all the time. We either:
a) ensure every document in the index is deleted before running each test
b) record every document we add and delete them after the test.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.