Logstash using incorrect address

[jarnold (Jack Arnold)]

Logstash appears to be using localhost:9200 to connect to my elasticsearch instance instead of its proper IP address, logstash error output here;

Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)
No Available connections
No Available connections
No Available connections
Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties

Strange, as it was working a month or two ago - Following a reboot it doesn't want to connect. Kibana & Elasticsearch appear to be working correctly.

I've checked my logstash.yml, pipelines.yml and my conf.d/pipeline.yml configs and not one of them contains localhost anywhere, they all use the private IP. Stranger still is the fact that Elasticsearch is set to bind to all interfaces (0.0.0.0 in the config) and logstash resides on the same machine.

Running logstash with the --path.settings flag resolves the issue, however, I would like logstash to automatically default to these settings on startup - /etc/logstash/startup.options and the corresponding script do not appear to apply any changes to logstash's default settings.

Any ideas? Thanks.

[Badger]

This could be x-pack monitoring, which appears to always want to talk to localhost:9200. Similarly for the x-pack license check.

[jarnold (Jack Arnold)]

x-pack monitoring is not enabled in my logstash.yml file. Logstash is also exiting after I get the

Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)

error.

Weirdly enough it's also doing this when running the logstash binary with the --path.settings flag.

Another weird thing is as of this morning, logstash didn't want to run at all - I found the cause of this was the logstash-plain.log being owned by root:root instead of logstash:logstash. I'm not sure what could have caused this to change, as this server is only used by me, and I have not used it since posting the above question.

Any ideas what could cause this? I imagine the issue with curator is tied to this somehow