Elasticsearch on YARN problem to identify GLOBAL URI


(Gaetano Fabiano ®) #1

Dear All,
I'm new on elastic search, so I'm sorry in advance if I'm doing some concectual error.

Following this guide https://www.elastic.co/guide/en/elasticsearch/hadoop/current/ey-usage.html
I installed and configured elastic serch running on Yarn, but now I have some doubt about.

after running this command
sudo hadoop jar elasticsearch-yarn-2.2.0-rc1.jar -start containers=3 app.queue=alpha container.mem=512

if I go on my http://master-ip-address:8088/cluster
i can see the application running

application_1454674969620_0082 root Elasticsearch-YARN ELASTICSEARCH alpha Wed Feb 10 13:03:29 0100 2016 N/A RUNNING UNDEFINED 4 4 1792 29.2 23.3
ApplicationMaster 0

Only if I read on log i can obtain information about the rest uri of the elasticsearch REST API
Below the log row

[2016-02-10 12:04:19,993][INFO ][http ] [Amphibius] publish_address {cluster-machine1-address:9201}, bound_addresses {[::]:9201}
[2016-02-10 12:04:20,001][INFO ][node ] [Amphibius] started

Now if I use a client REST I am able to call the api using the following address
cluster-machine1-address:9201

the PROBLEM IS:
if I restart the elastic-searc jar on yarn the address change and have some like this
cluster-machine2-address:9201

There is a way to have a "kind of proxy manager" to invoke?
I mean I need a stable address to call.

Is there anybody that can help me.
Thanks in advance.
Gaetano


(Costin Leau) #2

Unfortunately not and one of the reasons why the YARN integration is BETA.
YARN does not provide any provisioning or network/ip guarantees, data/node affinity - basically support for long running services (whether it's ES, mysql, postgres, you name it).


(system) #3