How to setup ElasticSearch-Hadoop to work thru HTTPS?

(Marat Galeyev) #1


I'm setting up a elasticsearch-hadoop connector to pull data to hadoop. I wrote a mapper/reducer job for it and tested on my dev box, where elasticsearch set up on a default 9200 port at http://localhost:9200. However, the prod version is deployed at http://{url}/{prefix} and it's not clear how to properly set job settings in this case. I tried:

	conf.set("es.nodes", "{url}");
	conf.set("es.port", "443");
	conf.set("es.nodes.path.prefix", "{prefix}");

however, it fails with:

Exception in thread "main" Connection error (check network and/or proxy settings)- all nodes failed; tried [[]]

I also tried to specify:

	conf.set("es.nodes", "{url}");
	conf.set("es.port", "80");
	conf.set("es.nodes.path.prefix", "{prefix}");
	conf.set("", "true");

, however it fails with:

 Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot instantiate SSL - true SSLContext not available

What would be the right way to configure elasticsearch-hadoop connector to work with elasticsearch over https?

Thank you.

(Costin Leau) #2

So ES is deployed under http://somehost/someprefix/ right?

In this case why use es.port 443? Since you are using http it should be port 80. If you made a typo and used https then make sure to specify; true is an invalid setting.

Take a step back and check out the configuration page; it gives an example on what options you have available. For example, the SSL options are discussed here and security over all here.

Also the docs contain a section on troubleshooting which you might find handy.

(Marat Galeyev) #3

@costin, thank you very much for your response, it helped to solve the problem. Yes, there was a typo, I used https and also incorrectly used property.

(system) #4