Logstash with x-pack authentication problems without Authentication Header

Hello,
as stated in Logstash with x-pack authentication problems
I have the same problems after I have updated ELK-Stack with x-pack.

[2017-09-07T08:44:37,664][ERROR][logstash.outputs.elasticsearch] Got a bad response code from server, but this code is not considered retryable. Request will be dropped {:code=>401, :response_body=>"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"missing authentication token for REST request [/_bulk]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}}],\"type\":\"security_exception\",\"reason\":\"missing authentication token for REST request [/_bulk]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}},\"status\":401}"}

Logstash does not include BASIC Authorisations Headers.
I tried to tcpdump the communication and it is true:

POST /_bulk HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json
Content-Length: 94879 
Host: localhost:9200
User-Agent: Manticore 0.6.1
Accept-Encoding: gzip,deflate

{"index":{"_id":null,"_index":"metricbeat-2017.09.07","_type":"metricsets","_routing":null}}
. ....

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="security" charset="UTF-8"
content-type: application/json; charset=UTF-8
content-encoding: gzip
transfer-encoding: chunked

ba
............K
 .@.D.2.:.....P...!.DB;.fH.hw..B.

I tried the official docu and created roles and users, add the
xpack.monitoring.elasticsearch.username: "elastic"
xpack.monitoring.elasticsearch.password: "changeme"

in the logstash.yml file etc.

Also a basic curl works

 curl -v -u elastic:changeme http://localhost:9200

Is it a bug??????

What does your Logstash configuration look like?

Thank you for the quick reply.
Yes, I found the mistake.
THANK YOU very Much!!

In the logstash.yml I had a reference to

  path.config: /etc/logstash/conf.d 

which inside there was beats.conf file where I have not noticed that there also must be
the user and password inserted. Current output is:

output {
   if [@metadata][beat] {
     elasticsearch {
         hosts => ["http://localhost:9200"]
         index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
        user => "elastic"
        password => "changeme"
      }
   }
     else {
         elasticsearch {
               hosts => [ "127.0.0.1:9200" ]
               user => "elastic"
               password => "changeme"
         }
     }
 }

But why do I register the user both on beats.conf and the logstash,yml ?

The entry in logstash.yml specifies where Logstash will send monitoring data. Configuration in the config determines where data go.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.