Can't start newly created river on several active nodes

Elastic Stack Elasticsearch
1

I'm trying to implement non-stopping river/mapping update functionality for
my elasticsearch cluster.
I have two empty nodes with no rivers or indexes at all. I create new river
manually using river's API. New river appears in _river index but it has
only one document 'meta', and no river's thread started.
It happens time-to-time, not always.
What could be problem ?

Thank you!

2

You mean you implemented your own river?

On Mon, May 21, 2012 at 9:48 AM, Nikolai Muhhin nikolai.muhhin@gmail.comwrote:

I'm trying to implement non-stopping river/mapping update functionality
for my elasticsearch cluster.
I have two empty nodes with no rivers or indexes at all. I create new
river manually using river's API. New river appears in _river index but it
has only one document 'meta', and no river's thread started.
It happens time-to-time, not always.
What could be problem ?

Thank you!

3

+kimchy Yes, it implements interface River, using JDBC connection to fetch
delta data.
I have JdbcRiverPlugin which extends AbstractPlugin (source code is here
https://gist.github.com/2780166)
JdbcRiver should start new thread of OracleJdbcIndexer (it's code is here
https://gist.github.com/2780170)

Starting elastic on one single node creates river and fetches data fine.
But if two empty nodes started and I send request to create river, then
time to time river does not start. I can't find reason for that and exact
scenario to call this problem.

4

There can only exist one river per type per cluster. Only one node
should have a running river.

On Thu, May 24, 2012 at 1:17 AM, Nikolai Muhhin
nikolai.muhhin@gmail.com wrote:

+kimchy Yes, it implements interface River, using JDBC connection to fetch
delta data.
I have JdbcRiverPlugin which extends AbstractPlugin (source code is here
gist:2780166 · GitHub)
JdbcRiver should start new thread of OracleJdbcIndexer (it's code is
here gist:2780170 · GitHub)

Starting elastic on one single node creates river and fetches data fine. But
if two empty nodes started and I send request to create river, then time to
time river does not start. I can't find reason for that and exact scenario
to call this problem.

5

Hello everybody!

I have the similar problem:
I try to run two rivers consequentially on two-nodes cluster (CouchDB
rivers).

curl -XPUT 'localhost:9200/_river/my_db1/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot2","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

curl -XPUT 'localhost:9200/_river/my_db/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

The first river is started successfully. The second river cannot be
started. Only when I stop one of the nodes, the second river starts.

Did I do the things correctly ?

6

Hello!

If you look at http://www.elasticsearch.org/guide/reference/river/ on the "Cluster Allocation" part it says that rivers are singletons within a cluster. That's why you can create a single river of the same type, but not second one.

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch

Hello everybody!

I have the similar problem:

I try to run two rivers consequentially on two-nodes cluster (CouchDB rivers).

curl -XPUT 'localhost:9200/_river/my_db1/_meta' -d '{"type" : "couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" : "my_couch_db","filter" : null},

"index" : {"index" : "kot2","type" :

"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings": {"_id":{"type": "string","store": "yes"},"name": {"type": "string","store": "yes"}}}'

curl -XPUT 'localhost:9200/_river/my_db/_meta' -d '{"type" : "couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" : "my_couch_db","filter" : null},

"index" : {"index" : "kot","type" :

"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings": {"_id":{"type": "string","store": "yes"},"name": {"type": "string","store": "yes"}}}'

The first river is started successfully. The second river cannot be started. Only when I stop one of the nodes, the second river starts.

Did I do the things correctly ?

7

Ok, but it works when I use only ONE node.

On Tuesday, May 29, 2012 1:44:44 PM UTC+3, Rafał Kuć wrote:

Hello!

If you look at Elasticsearch Platform — Find real-time answers at scale | Elastic on the
"Cluster Allocation" part it says that rivers are singletons within a
cluster. That's why you can create a single river of the same type, but not
second one.

*--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch

Hello everybody!

I have the similar problem:
I try to run two rivers consequentially on two-nodes cluster (CouchDB
rivers).

curl -XPUT 'localhost:9200/_river/my_db1/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot2","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

curl -XPUT 'localhost:9200/_river/my_db/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

The first river is started successfully. The second river cannot be
started. Only when I stop one of the nodes, the second river starts.

Did I do the things correctly ?

8

I mean: several rivers with same type but different names (in RiverName
scope) can work in cluster that consists ONLY ONE node.
When cluster has more nodes - the river behavior is unpredictable (can be
started or cannot).

On Tuesday, May 29, 2012 1:58:27 PM UTC+3, Jaroslav Judin wrote:

Ok, but it works when I use only ONE node.

On Tuesday, May 29, 2012 1:44:44 PM UTC+3, Rafał Kuć wrote:

Hello!

If you look at Elasticsearch Platform — Find real-time answers at scale | Elastic on
the "Cluster Allocation" part it says that rivers are singletons within a
cluster. That's why you can create a single river of the same type, but not
second one.

*--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -
Elasticsearch

Hello everybody!

I have the similar problem:
I try to run two rivers consequentially on two-nodes cluster (CouchDB
rivers).

curl -XPUT 'localhost:9200/_river/my_db1/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot2","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

curl -XPUT 'localhost:9200/_river/my_db/_meta' -d '{"type" :
"couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :
"my_couch_db","filter" : null},
"index" : {"index" : "kot","type" :
"kot_db","bulk_size" : "100","bulk_timeout" : "10ms"},"mappings":
{"_id":{"type": "string","store": "yes"},"name": {"type": "string","store":
"yes"}}}'

The first river is started successfully. The second river cannot be
started. Only when I stop one of the nodes, the second river starts.

Did I do the things correctly ?