Logstash won't authenticate with elasticsearch

[jarnold (Jack Arnold)]

Hello,

I'm setting up a new stack & i'm having some trouble with my logstash authentication. I'm getting this error after startup;

Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
Got response code '401' contacting Elasticsearch at URL 'http://localhost:9200/_xpack'

Which repeats ad-infinitum.

My logstash.yml is as follows;

Pastebin link as it's too many characters otherwise

And my pipeline configuration is currently this;

input {
  beats {
    port => "5044"
  }
}

filter {
  grok {
    match => { "message" => "%{GREEDYDATA}"}
  }
}

output {
  elasticsearch {
    hosts => [ "172.19.32.154" ]
    user => "logstash_system"
    password => "password"
  }
}

I've tried using the elastic user as well as logstash_system. If I curl or browse to 172.19.32.154:9200/_xpack & use these credentials it works, no errors.

Disabling xpack security in elasticsearch resolves the issue - Obviously this is not ideal as I would like to use security.

One thing that seems odd is that logstash is attempting to connect to localhost:9200. This is fine as elasticsearch's host address is set to 0.0.0.0 but nowhere in my logstashs config is localhost specified, it's always stated as 172.19.32.154. This leads me to believe logstash isn't loading my config, despite getting the same error when using the --path.settings flag.

Thanks

[jarnold (Jack Arnold)]

Anyone?

I'm currently running without authentication, but this is not ideal. Help would be appreciated.

[jarnold (Jack Arnold)]

Bumping this again for visibility. From what I understand I only need my username and password in the output section of my pipeline. However this doesn't work. Any ideas?