Org.elasticsearch.indices.IndexMissingException: [_river] missing

vozerov · October 2, 2013, 2:24pm

Good day!

I have a problem... or misconfiguration

I have installed https://github.com/sksamuel/elasticsearch-river-redis
plugin and add configuration to it:

curl -XPUT 'http://localhost:9200/_river/redisriver/_meta' -d '{
"type": "redis",
"redis": {
"hostname": "localhost",
"port" : "6379",
"json" : false,
"messageField" : "content",
"channels": "test-channel"
},
"index": {
"name": "test-redis",
}
}'

And after stop/start elasticsearch I see the following errors:

1-73:0.90.5 vozerov$ elasticsearch -f
[2013-10-02 18:18:52,593][INFO ][node ] [Achebe]
version[0.90.5], pid[96444], build[c8714e8/2013-09-17T12:50:20Z]
[2013-10-02 18:18:52,593][INFO ][node ] [Achebe]
initializing ...
[2013-10-02 18:18:52,606][INFO ][plugins ] [Achebe] loaded
[river-redis], sites []
[2013-10-02 18:18:54,968][INFO ][node ] [Achebe]
initialized
[2013-10-02 18:18:54,968][INFO ][node ] [Achebe]
starting ...
[2013-10-02 18:18:55,080][INFO ][transport ] [Achebe]
bound_address {inet[/127.0.0.1:9300]}, publish_address
{inet[/127.0.0.1:9300]}
[2013-10-02 18:18:58,110][INFO ][cluster.service ] [Achebe]
new_master [Achebe][ThLe6gTURWSNcUn6ctixGw][inet[/127.0.0.1:9300]], reason:
zen-disco-join (elected_as_master)
[2013-10-02 18:18:58,137][INFO ][discovery ] [Achebe]
elasticsearch_vozerov/ThLe6gTURWSNcUn6ctixGw
[2013-10-02 18:18:58,151][INFO ][http ] [Achebe]
bound_address {inet[/127.0.0.1:9200]}, publish_address
{inet[/127.0.0.1:9200]}
[2013-10-02 18:18:58,151][INFO ][node ] [Achebe] started
[2013-10-02 18:18:58,174][DEBUG][action.get ] [Achebe]
[_river][0]: failed to execute [[_river][river-redis][_meta]: routing
[null]]
org.elasticsearch.indices.IndexMissingException: [_river] missing
at
org.elasticsearch.indices.InternalIndicesService.indexServiceSafe(InternalIndicesService.java:334)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:96)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:42)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:161)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-10-02 18:18:58,705][INFO ][gateway ] [Achebe]
recovered [2] indices into cluster_state
[2013-10-02 18:18:58,730][DEBUG][action.get ] [Achebe]
[_river][0]: failed to execute [[_river][river-redis][_meta]: routing
[null]]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0]
CurrentState[RECOVERING] operations only allowed when started/relocated
at
org.elasticsearch.index.shard.service.InternalIndexShard.readAllowed(InternalIndexShard.java:704)
at
org.elasticsearch.index.shard.service.InternalIndexShard.get(InternalIndexShard.java:415)
at
org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:167)
at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:101)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:103)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:42)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:161)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-10-02 18:18:58,732][WARN ][river.routing ] [Achebe] failed
to get/parse _meta for [river-redis]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0]
CurrentState[RECOVERING] operations only allowed when started/relocated
at
org.elasticsearch.index.shard.service.InternalIndexShard.readAllowed(InternalIndexShard.java:704)
at
org.elasticsearch.index.shard.service.InternalIndexShard.get(InternalIndexShard.java:415)
at
org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:167)
at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:101)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:103)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:42)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:161)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-10-02 18:18:58,803][DEBUG][action.get ] [Achebe]
[_river][0]: failed to execute [[_river][river-redis][_meta]: routing
[null]]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0]
CurrentState[RECOVERING] operations only allowed when started/relocated
at
org.elasticsearch.index.shard.service.InternalIndexShard.readAllowed(InternalIndexShard.java:704)
at
org.elasticsearch.index.shard.service.InternalIndexShard.get(InternalIndexShard.java:415)
at
org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:167)
at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:101)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:103)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:42)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:161)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-10-02 18:18:58,804][WARN ][river.routing ] [Achebe] failed
to get/parse _meta for [river-redis]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0]
CurrentState[RECOVERING] operations only allowed when started/relocated
at
org.elasticsearch.index.shard.service.InternalIndexShard.readAllowed(InternalIndexShard.java:704)
at
org.elasticsearch.index.shard.service.InternalIndexShard.get(InternalIndexShard.java:415)
at
org.elasticsearch.index.get.ShardGetService.innerGet(ShardGetService.java:167)
at org.elasticsearch.index.get.ShardGetService.get(ShardGetService.java:101)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:103)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:42)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:161)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
[2013-10-02 18:18:58,932][INFO
][org.elasticsearch.plugin.river.redis.RedisSubscriber] Starting redis
subscriber

But river plugin works well... So is it critical errors, or something else?
Can somebody explain it...

Merci, have a nice day

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Karol_Gwaj · October 10, 2013, 3:06pm

+1 Im getting this same error

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

spinscale · October 10, 2013, 3:47pm

Hey,

dont worry about it too much, the message is not criticial as the river is
started up anyway (it is just retrying shortly after and then the river
starts up successfully). Furthermore the stack trace should go away with
the next release, see

can you test it (maybe the 0.90 branch) and report back?

--Alex

On Thu, Oct 10, 2013 at 5:06 PM, Karol Gwaj karol@gwaj.me wrote:

+1 Im getting this same error

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

amorais · October 24, 2013, 3:13pm

Hi Alex,

I'm having the same problem and you're right when you say that the river starts up anyway. However, in my case even though it was a oneshot river it fires up a complete set of bulk requests and ends up duplicating the document count.

Here is the sequence:

"org.elasticsearch.indices.IndexMissingException: [_river] missing"
"failed to execute [[_river][clfb_anoeee20131024152941_river][_meta]: routing [null]]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0] CurrentState[RECOVERING] operations only allowed when started/relocated (...)"
"failed to get/parse _meta for [clfb_anoeee20131024152941_river]
org.elasticsearch.index.shard.IllegalIndexShardStateException: [_river][0] CurrentState[RECOVERING] operations only allowed when started/relocated"
"[clfb_anoeee20131024152941_river] starting JDBC river: URL [jdbc:mysql://localhost:3306/clfb_anoeee], driver [com.mysql.jdbc.Driver], strategy [oneshot], index [clfb_anoeee20131024152941]/[jdbc]"
"Sending mapping updated to master: index [_river] type [clfb_anoeee20131024152941_river]"

And then it repeats this step and it repeats "starting JDBC river" step (4) - I'm guessing that this is the reason the duplicates...

This is tbe repetition:

"Sending mapping updated to master: index [_river] type [clfb_anoeee20131024152941_river]"
"update_mapping [clfb_anoeee20131024152941_river] (dynamic)"
"[clfb_anoeee20131024152941_river] starting JDBC river: URL [jdbc:mysql://localhost:3306/clfb_anoeee], driver [com.mysql.jdbc.Driver], strategy [oneshot], index [clfb_anoeee20131024152941]/[jdbc]"

Do you have any suggestions as to how I might fix this? The problem only occurs on shutdown and occurs in all of our ES installations (0.90.5 - with latest JDBC-river versions).

       -- André