AWS Error:- Cannot detect ES version: - typically this happens if the network/Elasticsearch cluster is not accessible or when tar geting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'

[krishnaSH (Krishna)]

I am connecting Hive in AWS EMR instancewith Elasticsearch and firing the command (PFB) to create a table in hive:

 create external table abc(schema) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.resource' = 'indexname/typename', 'es.port' = '9200', 'es.nodes'='x.x.x.x', es.wan.mode', 'es.nodes.wan.only' = 'true' . 'es.node' = 'x.x.x.x');

I am getting the below error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when tar
geting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'

Note: I dont have firewall active. Amongst the solutions I found on internet, it tells me to check whether my ES nodes are able to communicate with hadoop. How can I check that?

Could anyone please enlighten me with any other solution ?
TIA.

[james.baiera]

@krishnaSH

... 'es.nodes'='x.x.x.x', es.wan.mode', 'es.nodes.wan.only' = 'true' . 'es.node' = 'x.x.x.x' ...

These settings don't seem to have correctly paired quotes; I'm assuming this is a copy paste error?

This is a fairly common error when running on a cloud environment. Almost all cloud providers that offer distributed computing make it so that their worker nodes are placed in a sandboxed network. This means that they can ONLY talk to each other and to the whitelisted hosts from the cloud platform. This is different for each cloud distribution, so check to make sure that wherever your ES cluster is running that the worker nodes from Hive are able to reach the cluster. It might help to reach out to your cloud provider to ask how to do this.