Race condition upon startup?

Elastic Stack Elasticsearch
4

Strange, can you gist a simple recreation? Btw, there is an index exists API
in 0.17.

On Mon, Aug 1, 2011 at 11:40 PM, Curtis Caravone caravone@gmail.com wrote:

Here's one example (this is using version 0.16.2):

org.elasticsearch.indices.IndexMissingException: [test_cache_directory]
missing
at
org.elasticsearch.cluster.metadata.MetaData.concreteIndices(MetaData.java:165)
at
org.elasticsearch.cluster.metadata.MetaData.concreteIndices(MetaData.java:132)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.(TransportBroadcastOperationAction.java:153)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction.doExecute(TransportBroadcastOperationAction.java:71)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction.doExecute(TransportBroadcastOperationAction.java:43)
at
org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:61)
at org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:47)
at
org.elasticsearch.client.node.NodeIndicesAdminClient.refresh(NodeIndicesAdminClient.java:195)
...

The Java code leading up to it is this (attempt to create index, wait for
yellow, refresh):

    try {

client.admin().indices().create(createIndexRequest(indexName).settings(settings)).actionGet();
} catch (IndexAlreadyExistsException e) {
logIndexExists(e, indexName);
} catch (RemoteTransportException r) {
Throwable t = r.getMostSpecificCause();
if (t instanceof IndexAlreadyExistsException) {
logIndexExists(t, indexName);
} else {
throw r;
}
}

client.admin().cluster().health(clusterHealthRequest(indexName).waitForYellowStatus()).actionGet();

client.admin().indices().refresh(refreshRequest(indexName)).actionGet();

I can try to extract a stand-alone example if you need it.

Curtis

On Mon, Aug 1, 2011 at 1:23 PM, Shay Banon kimchy@gmail.com wrote:

The status failure on specific shards can happen, they are not propagated
as a complete failure to the client using the API.

The other one you specify, the one with the yellow status and failing to
index, can you post a sample code for it?

On Mon, Aug 1, 2011 at 11:07 PM, Curtis Caravone caravone@gmail.comwrote:

Hi all,

I am getting intermittent failures about missing indices in my code which
uses the elasticsearch Java API. I can easily reproduce a similar exception
in a stand-alone elasticsearch instance using elasticsearch-head as follows:

  1. Start up an elasticsearch node that has an existing index (i.e. run
    <elasticsearch_home>/bin/elastcsearch -f)

  2. Immediately go to elasticsearch-head and quickly hit "Connect"
    repeatedly until yellow status appears

  3. The elasticsearch logs show an exception:

    [2011-08-01 12:49:07,217][DEBUG][action.admin.indices.status] [Armor]
    [test][3], node[y5QCD5imRzmfbUZcIhkMsQ], [P], s[INITIALIZING]: Failed to
    execute
    [org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@2f6e4ddd
    ]
    org.elasticsearch.index.IndexShardMissingException: [test][3] missing
    at
    org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:172)
    at
    org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:135)
    at
    org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:58)
    at
    org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:227)
    at
    org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:205)
    at
    org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:181)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)

I get a similar exception that fails my unit tests when I try to do index
operations on a local node immediately after waiting for yellow status.

The above stack trace is from 0.17.2, but I get the same behavior in
0.16.2 and 0.16.5. I'm pretty sure I never ran into this problem when I was
using 0.15.2.

Any ideas on what the problem may be or how to work around it?

thanks,

Curtis