How to setup ElasticSearch-Hadoop to work thru HTTPS?

Elastic Stack Elasticsearch
1

Hello,

I'm setting up a elasticsearch-hadoop connector to pull data to hadoop. I wrote a mapper/reducer job for it and tested on my dev box, where elasticsearch set up on a default 9200 port at http://localhost:9200. However, the prod version is deployed at http://{url}/{prefix} and it's not clear how to properly set job settings in this case. I tried:

	conf.set("es.nodes", "{url}");
	conf.set("es.port", "443");
	conf.set("es.nodes.path.prefix", "{prefix}");

however, it fails with:

Exception in thread "main" org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[10.205.12.97:443]]

I also tried to specify:

	conf.set("es.nodes", "{url}");
	conf.set("es.port", "80");
	conf.set("es.nodes.path.prefix", "{prefix}");
	conf.set("es.net.ssl.protocol","true");
	conf.set("es.net.ssl", "true");

, however it fails with:

 Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot instantiate SSL - true SSLContext not available
	at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:163)

What would be the right way to configure elasticsearch-hadoop connector to work with elasticsearch over https?

Thank you.

2

So ES is deployed under http://somehost/someprefix/ right?

In this case why use es.port 443? Since you are using http it should be port 80. If you made a typo and used https then make sure to specify es.net.ssl;es.net.ssl.protocol true is an invalid setting.

Take a step back and check out the configuration page; it gives an example on what options you have available. For example, the SSL options are discussed here and security over all here.

Also the docs contain a section on troubleshooting which you might find handy.

1 Like
3

@costin, thank you very much for your response, it helped to solve the problem. Yes, there was a typo, I used https and also incorrectly used es.net.ssl.protocol property.