There seem to be quite a few ssl related config parameters but unfortunately with little explanation as to what is needed and what is not. I have an ES cluster behind an SSL vip and each node in the cluster is protected at the host level with nginx that requires authentication.
The set up works fine from standalone programs running on remote clients so I can query, post and such to the cluster. But having no luck with ES-Hadoop. I must be missing some key config to the ES-Hadop set up with SSL/auth & proxies and such. Here is my set up.
The vip/LB is like
It routes the incoming requests to any one of the esnodes in its pool (esnode_1 thru esnode_N) like:
https://esnode_N:xxx
The above request is fielded by nginx that authenticates the request and routes it locally to
http://localhost:yyy
where elasticsearch is running
So what combination of parameters in ES-Hadoop do I need to set to get it all to work nicely under ES-Hadoop?
I have standalone programs that work fine to write/read from this protected ES cluster, so I know that the set up is sound. I have gotten it to work with setting
es.nodes.wan.only => true
es.net.ssl: => true
But that is bad for performace!
I have tried a variety of combinations trying to avoid the 'wan' option but no dice...
es.nodes: the vip ??? Or one of the actual nodes ???
es.port: 443 ??? Or xxx where nginx resolves & authenticates the request on the host, Or yyy where ES actually runs on the host ???
es.net.http.auth.user : service_user
es.net.http.auth.pass: service_user_password
es.net.proxy.https.user: service_user ??? this or the above?
es.net.proxy.https.pass: service_user_password ??? this or the above?
es.net.ssl: true ??? Only the proxy & nginx access are ssl... ES is http ...
Given that ES-Hadoop tries to talk directly to the shard nodes but access is limited to localhost only or througb the vip... and the port numbers & host/server/vip names are all different
Is ES-Hadoop even supposed to work for situations like these, other than using the 'wan' option?
Thanks