I'm testing Logstash sniffing in our environment for the Elasticsearch output. So far, it appears to resolve a few issues I've had, but I've got a few questions:
sniffing => true in my elasticsearch output appears to return all elasticsearch nodes in my cluster. I use dedicated ingest nodes. When Logstash is choosing a node to send output to, does it only choose from among the ingest nodes? If not, is there a setting I can include in
sniffing_path to force it to only return ingest nodes?
How frequently does Logstash do an "update" sniff, to update the list of nodes? Or do I have to restart Logstash after adding/removing nodes to/from my cluster?
The default for the sniffing_delay option on the output is documented as 5 seconds, but looking at the code it appears to be 10. Down at line 152 you can see it just sits in a loop sniffing and then delaying.
Looking at the sniffing code it skips "master" nodes and includes everything else.
I saw that, but couldn't determine from the documentation whether it sniffed every 5s, or that was just a "sniff again in 5s if you don't get a response."
Thanks for pointing me there. Unfortunately, that's a bummer, as I use dedicated ingest nodes. However, I'm not seeing errors so far.
I know that the
sniffer_path allows me to set a specific endpoint including, presumably, parameters, if accepted. Unfortunately, the documentation for the node info API in Elasticsearch makes it appear that this is not possible. And while
_nodes/_master/http will return master node info, there doesn't appear to be a corresponding
_nodes/_ingest/http to return ingest node info.
Is there anything you may be aware of that I'm missing?
No, but I have not run elasticsearch for years, and when I did all my nodes were the same, so it is not something I know a lot about.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.