Can anyone explain this exception

Elastic Stack Elasticsearch
1

I sometimes get the following exception when running one of my
unittests against ES. The exception is returned to me in a
ActionListener.onFailure object that I give as a
parameter when I call execute on SearchRequestBuilder. I never get the
exception when running the test on my local Mac, but when the test is
run on our TeamCity machine (Ubuntu) the exception occurs 90% of the
time.

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
state not recovered / initialized];[2/no master];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedException(ClusterBlocks.java:
131)
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(ClusterBlocks.java:
115)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction.(TransportSearchTypeAction.java:116)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.(TransportSearchQueryThenFetchAction.java:72)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.(TransportSearchQueryThenFetchAction.java:63)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:
60)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:
52)
at
org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:
61)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:
112)
at
org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:
49)
at
org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:
61)
at org.elasticsearch.client.node.NodeClient.search(NodeClient.java:
186)
at
org.elasticsearch.client.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:
589)
at
org.elasticsearch.client.action.support.BaseRequestBuilder.execute(BaseRequestBuilder.java:
56)
at ... my junit-test code ...

The test is running 10 threads doing indexing (inserting) and
reindexing (updating) the on "the same" documents in ES, so it relying
on the "optimistic locking based on versioning" and "unique constraint
on _id" to work perfectly (they probably are, but I am not able to
interpret the exception here).

Regards, Per Steffensen

2

I imagine that it might be because the node hasnt completely started
before I start using it. I do the following in a @Before in my
unittest:
Node node = nodeBuilder()
.local(true)
.settings(ImmutableSettings.settingsBuilder()
.put("index.number_of_shards", shards)
.put("index.number_of_replicas", replicas).build())
.build()
.start();

Can I somehow wait for the node to be completely started? Looking at
the code I do not see a way to do that?

Regards, Per Steffensen

3

Wait for yellow would probably too expensive for every @Before

So I suggest to do node initialization in @BeforeClass and then put a
deleteIndex in @Before

In my unit tests I don't even need to wait for yellow state

On 29 Sep., 14:00, Steff st...@designware.dk wrote:

I imagine that it might be because the node hasnt completely started
before I start using it. I do the following in a @Before in my
unittest:
Node node = nodeBuilder()
.local(true)
.settings(ImmutableSettings.settingsBuilder()
.put("index.number_of_shards", shards)
.put("index.number_of_replicas", replicas).build())
.build()
.start();

Can I somehow wait for the node to be completely started? Looking at
the code I do not see a way to do that?

Regards, Per Steffensen

4

I know it is not what Karussell suggested, but now I have the
following in my unittest class, and it never fails on the first test
but often on the second test. Guess it is due to the node not finished
its close before another note is started. Can anyone be more detailed
about what to do in @BeforeClass, @AfterClass, @Before and @After in
unittests to write good stable tests against ES. Thanks!

@Before
public void before()
{
    node = Node node = nodeBuilder()
    .local(true)
    .settings(ImmutableSettings.settingsBuilder()
            .put("index.number_of_shards", 1)
            .put("index.number_of_replicas", 0).build())
    .build()
    .start();
}

@After
public void after()
{
	for (String indexName : createdIndices) {

client.admin().indices().delete(Requests.deleteIndexRequest(indexName)).actionGet();
}

    if (node != null)
    {
        node.close();
        node = null;
    }
}

Karussell suggests to wait for yellow to be sure that the node has
completely started. But waiting for yellow requires an index.
Basically I just want to know when the node has completely started -
even though there might be no indexes yet.

Regards, Per Steffensen

On 29 Sep., 15:38, Karussell tableyourt...@googlemail.com wrote:

Wait for yellow would probably too expensive for every @Before

https://github.com/karussell/Jetwick/blob/master/src/main/java/de/jet...

So I suggest to do node initialization in @BeforeClass and then put a
deleteIndex in @Before

In my unit tests I don't even need to wait for yellow state

On 29 Sep., 14:00, Steff st...@designware.dk wrote:

I imagine that it might be because the node hasnt completely started
before I start using it. I do the following in a @Before in my
unittest:
Node node = nodeBuilder()
.local(true)
.settings(ImmutableSettings.settingsBuilder()
.put("index.number_of_shards", shards)
.put("index.number_of_replicas", replicas).build())
.build()
.start();

Can I somehow wait for the node to be completely started? Looking at
the code I do not see a way to do that?

Regards, Per Steffensen

5

On 30 Sep., 09:03, Steff st...@designware.dk wrote:

I know it is not what Karussell suggested, but now I have the
following in my unittest class, and it never fails on the first test
but often on the second test. Guess it is due to the node not finished
its close before another note is started. Can anyone be more detailed
about what to do in @BeforeClass, @AfterClass, @Before and @After in
unittests to write good stable tests against ES. Thanks!

@Before
public void before()
{
    node = Node node = nodeBuilder()
    .local(true)
    .settings(ImmutableSettings.settingsBuilder()
            .put("index.number_of_shards", 1)
            .put("index.number_of_replicas", 0).build())
    .build()
    .start();
}

@After
public void after()
{
    for (String indexName : createdIndices) {

client.admin().indices().delete(Requests.deleteIndexRequest(indexName)).actionGet();
}

    if (node != null)
    {
        node.close();
        node = null;
    }
}

Karussell suggests to wait for yellow to be sure that the node has
completely started. But waiting for yellow requires an index.
Basically I just want to know when the node has completely started -
even though there might be no indexes yet.

Regards, Per Steffensen

Forgot to say that the tests themselves create the indices they need
and add them to the list createdIndices, so that they will be deleted
in after(). The tests use the following code to create the indices
they need:

private void createIndex(String indexName) {
	if (!createdIndices.contains(indexName)) {
    	String mappingDescription = "{\"" + INDEX_TYPE + "\" : {" +

""properties" : {" + getESMappingStr() + "}}}";
AdminClient admin = client.admin();
Settings settings =
ImmutableSettings.settingsBuilder().put("number_of_shards",
1).put("number_of_replicas", 0).build();
CreateIndexRequest request =
Requests.createIndexRequest(indexName).settings(settings).mapping(INDEX_TYPE,
mappingDescription);
CreateIndexResponse result =
admin.indices().create(request).actionGet();
// wait for ready
admin.cluster().health(new
ClusterHealthRequest(indexName).waitForYellowStatus()).actionGet();
createdIndices.add(indexName);
}
}

Regards, Per Steffensen

6

My problem seemed to be due to the fact that all the threads in the
test used their own node/client. I have changed so that they all share
the same node/client, and now the test is constantly green. Guess
conclusion is that you should not have more than one node/client in
the same JVM :slight_smile:

Regards, Per Steffensen

7

You can have more than one instance of a local node in the same JVM, but,
they will discover each other (within the same classloader, using static
vars). I am not sure what your tests do, but if you need them to use
different "clusters", you can use a different cluster name for the node(s)
you start per test.

On Fri, Sep 30, 2011 at 5:17 PM, Steff steff@designware.dk wrote:

My problem seemed to be due to the fact that all the threads in the
test used their own node/client. I have changed so that they all share
the same node/client, and now the test is constantly green. Guess
conclusion is that you should not have more than one node/client in
the same JVM :slight_smile:

Regards, Per Steffensen