Logstash Monitoring Error

Elastic Stack Logstash
1

My ElasticStack decided to go haywire today and I really have no idea what went wrong. After reinstalling the services, Logstash now logs this error message non-stop:

[2018-04-02T19:33:52,358][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://logstash_system:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused: connect"}

My logstash.yml looks like this:

 node.name: Server.FQDN
 path.data: D:/ElasticStack/Data/LogStash
 config.reload.automatic: true
 config.reload.interval: 5s
 pipeline.workers: 12
 pipeline.batch.size: 250
 http.host: "192.168.1.1"
 http.port: 9600
 xpack.monitoring.elasticsearch.username: logstash_system
 xpack.monitoring.elasticsearch.password: logstash
 slowlog.threshold.warn: 2s
 slowlog.threshold.info: 1s
 slowlog.threshold.debug: 500ms
 slowlog.threshold.trace: 100ms
# log.level: debug
 path.logs: D:/ElasticStack/Logs/LogStash

My Elasticsearch.yml looks like this:

cluster.name: ElasticStack
node.name: Server.FQDN
path.data: D:/ElasticStack/Data/ElasticSearch
path.logs: D:/ElasticStack/Logs/ElasticSearch
bootstrap.memory_lock: true
network.host: 192.168.1.1
http.port: 9200

As expected, monitoring in Kibana for Logstash does not appear. Why is the monitoring engine trying to connect on localhost?

2

The default values for xpack.monitoring.elasticsearch.* are for localhost, so if it fails to find specific configurations in your logstash.yml, it will fall back to the default values.

My guess is that your uninstall/reinstall removed the config yaml and we're back to the default config that ships with Logstash.

3

Either change Elasticsearch to listen on all IPs (which would include localhost) by setting network.host: 0.0.0.0 in elasticsearch.yml.

OR...

Tell Logstash monitoring where to find Elasticsearch by adding the following to logstash.yml:

xpack.monitoring.elasticsearch.url: [ "http://192.168.1.1:9200" ]

The latter would also be required if Elasticsearch and Logstash on separate nodes.

4

I only uninstalled the Windows services, never removed the actual files. The configs posted above were direct copy/paste of what's there.

I actually had it set to 0.0.0.0 originally. I changed it to 192.168.1.1 to try and force a specific interface to listen on. I will add the xpack.monitoring config line and give that a shot.

5

Setting xpack.monitoring.elasticsearch.url fixed it for me. Thanks.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.