Questions re Sniffing in Logstash elasticsearch output

Elastic Stack Logstash
#1

I'm testing Logstash sniffing in our environment for the Elasticsearch output. So far, it appears to resolve a few issues I've had, but I've got a few questions:

  1. sniffing => true in my elasticsearch output appears to return all elasticsearch nodes in my cluster. I use dedicated ingest nodes. When Logstash is choosing a node to send output to, does it only choose from among the ingest nodes? If not, is there a setting I can include in sniffing_path to force it to only return ingest nodes?

  2. How frequently does Logstash do an "update" sniff, to update the list of nodes? Or do I have to restart Logstash after adding/removing nodes to/from my cluster?

#2

The default for the sniffing_delay option on the output is documented as 5 seconds, but looking at the code it appears to be 10. Down at line 152 you can see it just sits in a loop sniffing and then delaying.

Looking at the sniffing code it skips "master" nodes and includes everything else.

#3

I saw that, but couldn't determine from the documentation whether it sniffed every 5s, or that was just a "sniff again in 5s if you don't get a response."

Thanks for pointing me there. Unfortunately, that's a bummer, as I use dedicated ingest nodes. However, I'm not seeing errors so far.

I know that the sniffer_path allows me to set a specific endpoint including, presumably, parameters, if accepted. Unfortunately, the documentation for the node info API in Elasticsearch makes it appear that this is not possible. And while _nodes/_master/http will return master node info, there doesn't appear to be a corresponding _nodes/_ingest/http to return ingest node info.

Is there anything you may be aware of that I'm missing?

#4

No, but I have not run elasticsearch for years, and when I did all my nodes were the same, so it is not something I know a lot about.

1 Like