Undesired client nodes discovered via sniffing


(Jimmy Jones) #1

I've been following the Kibana guide which suggests in production to set up an Elasticsearch client node for Kibana to use. However, by default any ingest processes using sniffing will start sending indexing requests via this node as well, which seems undesirable for a number of reasons, such as your webserver isn't as powerful as "normal" Elasticsearch nodes.

One possible workaround is to set http.host: "127.0.0.1", but if a sniffing client happens to have a node for a different cluster running on the same machine for whatever reason, it could ingest into the wrong cluster and cause much confusion. Should it possible to setup a "private" node, for this kind of scenario, which doesn't advertise its http address to the rest of the cluster?


(Ed) #2

I am not sure what you mean by sniffing. as ES does not consume files that is usually reserved for Logstash.

But I think the rest of what your talking about is this

https://www.elastic.co/guide/en/kibana/current/production.html

A ES node can be MASTER, DATA or NEITHER (Or any combination ) depending on your architecture.

The node running on the WEB server as you say, I think would follow the above doc to have
In step 3

node.master: false
node.data: false

(Jimmy Jones) #3

Sniffing is used by clients to discover other nodes in the cluster they can use, for example python and logstash.

However some nodes in the cluster, such as a client node running on a webserver, may be unsuitable, and I think it would be useful to have a way to avoid them.


(Ed) #4

Oh your talking about the Zen discovery . In a production environment it recommended to be turned off and to define the servers in the cluster.

Well yes if a server was unsuitable to run elasticsearch in a "Master" "data" or Neither role you should not run one there.

Every server to the function it needs to run. I mean a ES Node set to False False does nothing but accept a request and send it to all the nodes in the cluster and aggregates the final result that sends back to you.

Master and data nodes require more "resources" and you should avoid putting them on low resource or servers where it could impact your users.if you have thin resources.


(Jimmy Jones) #5

Not Zen discovery - that is for nodes in the cluster to discover each other; sniffing in this case is where a HTTP client discovers the rest of the nodes in the cluster it can use given one or multiple seed nodes. However this will discover all nodes, which in the case of someone running a client node on a webserver, is unwanted.


(Ed) #6

I was just looking at "Shutting down" a node gracefully and found this , and thought it would be of interest to you

There is a way for you to "tag" and "group" nodes with in a cluster and maniplate the routing of the Shards.

While I am not 100% versed in the concept it might provide a beginning for you

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-allocation.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html


(system) #7