Logstash got response code 401 contacting elasticsearch url

Elastic Stack Logstash
1

Hi everyone, i configured logstash pipeline and yml, from logs I can see that I receive something but when logstash tries to connect to elasticsearch it fails giving me

[2022-10-20T08:26:09,740][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://es01:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/'"}
[2022-10-20T08:26:09,756][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Got response code '401' contacting Elasticsearch at URL 'https://es01:9200/_xpack'"}

I tried to curl with

curl --cacert ./path-to-cert.crt -u user:pwd https://es01:9200

and it works perfectly fine, as I receive

{
"name" : "es01",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "mbfmRyDnRMCXZxifDx8McQ",
"version" : {
"number" : "8.4.1",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "2bd229c8e56650b42e40992322a76e7914258f0c",
"build_date" : "2022-08-26T12:11:43.232597118Z",
"build_snapshot" : false,
"lucene_version" : "9.3.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}

logstash.conf:

input {
udp {
port => "5044"
}
}
output {
elasticsearch {
hosts => ["https://es01:9200"]
ssl => true
ssl_certificate_verification => true
cacert => "/path-to-ca.crt"
user => "user"
password => "pwd"
index => "nas-%{+YYYY.MM.dd}"
}
}

logstash.yml:

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "https://es01:9200" ]
xpack.monitoring.elasticsearch.ssl.certificate_authority: "/path-to-ca.crt"

let me know if you need the docker-compose I used to create all containers, I really don't know what to do anymore since with curl I am able to connect to elasticsearch

2

Hello,

HTTP 401 means it's not authorized so probably look for missconfiguration of authentication parameters in conf.

If the curl request work fine this is related to your logstash config.

Can you maybe provide DEBUG logs in case something strange is going on under the hood ?

1 Like
3

the fact is that i copied the parameters for the curl directly from the config file, so it is impossible they are different.
how do I provide you debug logs?

4
1 Like
5

I just have this one

Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=>"nas-%{+YYYY.MM.dd}"}

and this one

[2022-10-20T09:59:07,157][DEBUG][logstash.outputs.elasticsearch][main] Found existing Elasticsearch template, skipping template management {:name=>"ecs-logstash"}

that i think are worth of notice, found nothing at all on authorization

6

if i try to do a curl:

curl --cacert ./path-to-cert.crt -u user:password https://es01:9200/_xpack

this is what i receive

{"build":{"hash":"--------",
"date":"2022-08-26T12:11:43.232597118Z"},
"license":{"uid":"----------",
"type":"basic","mode":"basic","status":"active"},
"features":{"aggregate_metric":{"available":true,"enabled":true},
"analytics":{"available":true,"enabled":true},
"archive":{"available":false,"enabled":true},
"ccr":{"available":false,"enabled":true},
"data_streams":{"available":true,"enabled":true},
"data_tiers":{"available":true,"enabled":true},
"enrich":{"available":true,"enabled":true},
"eql":{"available":true,"enabled":true},
"frozen_indices":{"available":true,"enabled":true},
"graph":{"available":false,"enabled":true},
"ilm":{"available":true,"enabled":true},
"logstash":{"available":false,"enabled":true},
"ml":{"available":false,"enabled":true},
"monitoring":{"available":true,"enabled":true},
"rollup":{"available":true,"enabled":true},
"searchable_snapshots":{"available":false,"enabled":true},
"security":{"available":true,"enabled":true},
"slm":{"available":true,"enabled":true},
"spatial":{"available":true,"enabled":true},
"sql":{"available":true,"enabled":true},
"transform":{"available":true,"enabled":true},
"voting_only":{"available":true,"enabled":true},
"watcher":{"available":false,"enabled":true}},
"tagline":"You know, for X"}

it could be that "logstash":{"available":false,"enabled":true}, the problem? I just noticed it, but i really don't know how to change that

7

Finally i found what was missing
inside logstash.yml you have to put xpack.monitoring.elasticsearch.username and xpack.monitoring.elasticsearch.password .
I don't know why these fields are not specified in logstash.yml | Logstash Reference [8.4] | Elastic

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.