# Using Nginx Proxy + ES + Spark **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [July 4, 2016, 10:44am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634 "2016-07-04T10:44:51Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Brad\_Jungsu\_Heo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brad_jungsu_heo/32/13135_2.png) [@Brad\_Jungsu\_Heo](https://discuss.elastic.co/u/Brad_Jungsu_Heo)\ **Post date:** [July 4, 2016, 10:44am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/1 "2016-07-04T10:44:51Z")
I'd like to setup Nginx proxy to secure my ES Cluster. But I'm stuck with setting Nginx proxy + ES + Spark. I bound `network.host` to `_local_` to prevent connecting directly without proxy. ```auto http.port: 9200 transport.tcp.port: 9300 network.host: _local_ transport.host: _non_loopback:ipv4_ ``` And configuration of nginx is below (nothing different from the es manual): ```auto server { listen 19200; server_name localhost; location / { proxy_pass http://localhost:9200/; proxy_http_version 1.1; proxy_set_header Connection "Keep-Alive"; proxy_set_header Proxy-Connection "Keep-Alive"; } } ``` Every node has same settings like belows and All ES plugins (head/marvel/kibana) and my python script worked well after adding proxy. ```auto +------------------+ +------------------+ +------------------+ |Node 1 | |Node 2 | |Node n | +------------------+ +------------------+ +------------------+ | Nginx | | Nginx | . | Nginx | |(listen on 19200) | |(listen on 19200) | . |(listen on 19200) | | | | | . | | | Elasticsearch | | Elasticsearch | | Elasticsearch | |(listens on 9200) | |(listens on 9200) | |(listens on 9200) | +------------------+ +------------------+ +------------------+ ``` I've changed Spark configuration like this (without nginx proxy my spark worked well): ```auto conf.set("spark.driver.allowMultipleContexts", "true") conf.set("es.index.auto.create", "true") conf.set("es.nodes.discovery", "true") conf.set("es.nodes", "es_hostname:19200") ``` But after setting a proxy, I'm getting an Exception `org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException` I enabled a `TRACE` logging level and I realized that Spark tried to connect `127.0.0.1:9200` or `public_ip:9200`. That's why I'm getting an `NoNodesLeftException`. After reading [ES Hadoop Configuration], I found `es.net.proxy.http.host` and `es.net.proxy.http.port`. These settings made spark work well. But the problem is that ONLY the proxy node are receiving `_bulk` requests, so it's incoming traffic is very high. ## Question. Can I conclude that `es.net.proxy.http.(host,port)` is the only way to use (proxy + ES + Spark)? Is there another way to use nginx proxy and Spark? I'm using ES 2.3.2 and es hadoop 2.3.2. Thanks. ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [July 11, 2016, 10:26pm UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/2 "2016-07-11T22:26:29Z")
Hello! To answer your question, I have to touch on node discovery and traffic very quickly. When es-hadoop bootstraps, it attempts to discover the nodes in your Elasticsearch cluster by executing a REST call to the `/_nodes/transport` endpoint. This returns a list of the cluster nodes and the addresses from which you can reach them. The connector will attempt to communicate with each node using their returned `http_address`. Because you have set the `network.host` property to be localhost, each node will say that localhost is the only address for HTTP requests. This has to do with how the _bind_ and _publish_ addresses are set up for the _HTTP_ module. Generally speaking, the host settings in Elasticsearch can be intimidating, but they [provide you with a lot of control over which interfaces your cluster uses for traffic](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html). Elasticsearch deals with two types of network traffic: _[Transport](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html)_ and _[HTTP](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html)_. There are two types hosts for each of these traffic types: `*.bind_host` and `*.publish_host`. They're normally the same values for both, but when working with proxies, you may have to make some changes to them. Now, to get back to the issue at hand: You have the `network.host` configured to use the local loopback and you have the `transport.host` configured to use a non loopback ip address. This makes sense because you probably don't want Nginx to get in the way between nodes in your cluster. When a node starts up, it chooses the address in `transport.host` to bind to and publish for _Transport_ traffic. If this were unset, it would default to `network.host`. You do not have any specific http settings, so the node defaults to `network.host` to bind to and publish for _HTTP_ requests. If you want ES-Hadoop to correctly discover the nodes, you'll need to configure the `http.publish_host` to be the address of the Nginx proxy you want the client to go through. When the node starts up, it will correctly ignore all incoming HTTP traffic not coming from localhost since the `http.bind_host` will default to the `network.host` property value. On the other side, when broadcasting its address to clients, it will return the value set for `http.publish_host` (the Nginx proxy address). This way, when the node discovery takes place, the nodes endpoint will return the address of the proxy for the connector to hit instead of just localhost. When following this pattern, you probably don't need the other proxy settings for your es-hadoop configuration. Now, whether or not this proxy will afford any true security is a completely different discussion. In this specific case, anyone would be able to connect to the cluster via the regular old Java Transport client since the _Transport_ traffic is un-policed. To be completely honest, if you're looking to implement serious access control to Elasticsearch, I would highly recommend the Shield Plugin instead. I hope this helps! ---
**Author:** ![Brad\_Jungsu\_Heo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brad_jungsu_heo/32/13135_2.png) [@Brad\_Jungsu\_Heo](https://discuss.elastic.co/u/Brad_Jungsu_Heo)\ **Post date:** [July 12, 2016, 4:10am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/3 "2016-07-12T04:10:18Z")
Thank you for the great help and support. I'll try `http.publish_host`. BTW, > In this specific case, anyone would be able to connect to the cluster via the regular old Java Transport client since the Transport traffic is un-policed. Does this mean that indices might be deleted via old Java Transport? What I want to do is preventing indices from being deleted. So I thought that Nginx would be a simple and nice solution as [Playing HTTP Tricks with Nginx](https://www.elastic.co/blog/playing-http-tricks-nginx) and [NGINX & Elasticsearch: Better Together](https://www.nginx.com/blog/nginx-elasticsearch-better-together/) said. ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [July 12, 2016, 2:03pm UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/4 "2016-07-12T14:03:02Z")
> [@Brad\_Jungsu\_Heo](#): > > Does this mean that indices might be deleted via old Java Transport? That is true. While the Transport client will eventually be deprecated for general client use, it is still a valid and accessible java package for most distributions. When the nodes start up, they bind their transport module to an externally available IP address. Anyone with a transport client could connect to the cluster, obtain an admin client, and delete which ever indices they so wish. This is why I recommend Shield, as the provided access control functionality is within Elasticsearch, and much harder to circumvent. ---
**Author:** ![Brad\_Jungsu\_Heo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brad_jungsu_heo/32/13135_2.png) [@Brad\_Jungsu\_Heo](https://discuss.elastic.co/u/Brad_Jungsu_Heo)\ **Post date:** [July 13, 2016, 7:00am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/5 "2016-07-13T07:00:53Z")
Hi. James. This is the test result of `http.publish_host` you've told me. I'm running ES (9200 port) and nginx (19200 port) on the same machine, so I've changed like this: ``` http.port: 9200 transport.tcp.port: 9300 network.host: _local_ http.publish_host: _non_loopback:ipv4_ <= this has been added transport.host: _non_loopback:ipv4_ ``` With these settings, Spark tried to connect `non_loopback:9200` but the problem is that`9200` port is only bound on `localhost` So `NoNodesLeftException` throwed. If ES and nginx run on different boxes, I think Spark works well. But in order to that, `network.host` should be bound on `non_loopback` which means anyone can execute REST APIs directly on ES without nginx (not secure). That's why I installed ES and nginx on the same box. Conclusion. I hope ES would have something like `http.publish_port` which is not supported yet. Thank you for your help. ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [July 13, 2016, 7:15am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/6 "2016-07-13T07:15:53Z")
> [@Brad\_Jungsu\_Heo](#): > > Conclusion. I hope ES would have something like http.publish\_port which is not supported yet. That should be a property that is available... Please see this documentation: [HTTP | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html) ---
**Author:** ![Brad\_Jungsu\_Heo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brad_jungsu_heo/32/13135_2.png) [@Brad\_Jungsu\_Heo](https://discuss.elastic.co/u/Brad_Jungsu_Heo)\ **Post date:** [July 13, 2016, 7:39am UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/7 "2016-07-13T07:39:40Z")
Oops. I didn't know that because I was only reading [Network Settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html) and it was not listed on there. Thanks a lot! 감사합니다! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:23pm UTC](https://discuss.elastic.co/t/using-nginx-proxy-es-spark/54634/8 "2017-07-06T13:23:57Z")