Here's a few tips I can recommend for running lots of automated integration
tests in quick succession against Elasticsearch. In my experience, its
point 1 that will easily overcome your problem, the other 2 are
nice-to-have's.
-
Set test indices to only have one shard. I have achieved this by
deploying an index template in my ES nodes that matches on index name with
the following pattern 'test*'. (ES docs for templates :
Elasticsearch Platform — Find real-time answers at scale | Elastic).
See example template below.
-
Set the data store to be 'memory' for test indices (I don't need to care
about actually hitting disc as I'm going to delete the test index when the
test finishes anyway). Again I do this via an index template, and match
the index name with the following pattern: inmemory. This will just ease
the strain on my ES node when running thousands of automated tests. See
example template below.
-
For each add/update query during a test, I set the refresh=true flag on
the index request, this forces ES to refresh its index immediately.
Without this, I ran in to timing issues due to the index refresh interval
(1 second by default) in ES. i.e. inserting some test data at the start of
the test, then immediately searching for it, not finding it because the
index hadn't refreshed yet, causing the test assertions to fail. Note, you
really don't want to be doing refreshing like this in production obviously.
{
"test_index_template": {
"template": "test*",
"order" : 1,
"settings": {
"index.number_of_shards": 1,
"index.number_of_replicas": 1
}
}
}
{
"inmemory_index_template": {
"template": "inmemory",
"order" : 99,
"settings": {
"index" : {
"store" : {
"type" : "memory"
},
"gateway" : {
"type" : "none"
}
}
}
}
}
On Monday, June 10, 2013 8:02:04 AM UTC+1, August Lilleaas wrote:
Correct 
I actually tried adding a wait for "green" immediately after
index creation, and noticed that the state of the cluster is always
"yellow" in my test run. And by always, I mean that the test always times
out (at the default 30s) since the state never seems to turn green. The
state is also yellow deterministically (all the time) as opposed to my
error (which occurs in about 1 /10 of the test runs).
FYI, I'm creating and deleting a lot of indexes. I'm basically creating
one index in elasticsearch for each of my individual tests in setUp.
Perhaps elasticsearch isn't designed for this? I suppose I could use one
index and a "foreign key" instead of a separate index for each of my top
level domain objects.
On Monday, June 10, 2013 8:39:19 AM UTC+2, Alexander Reelsen wrote:
Hey,
guess: Are you creating an index in your tests. Are you waiting until the
index and each shard of it is started? See more at
Elasticsearch Platform — Find real-time answers at scale | Elastic
--Alex
On Sun, Jun 9, 2013 at 11:39 PM, August Lilleaas augustl...@gmail.comwrote:
Hey folks,
when I run my test suite, I get the following error for a very specific
test, but not always (seems to be completely undeterministic when I get it):
SearchPhaseExecutionException[Failed to execute phase [query], total
failure; shardFailures {[na][my-index][3]: No active
shards}{[na][my-index][4]: No active shards}{[na][my-index][1]: No
active shards}{[na][my-index][2]: No active shards}{[na][my-index][0]:
No active shards}]
When this happens, there are no errors present in the elasticsearch logs.
Any suggestions on steps I can take to debug this problem?
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.