Monitoring User for Logstash

Hi guys,

I took over an elastic cluster with 3 nodes. All of the nodes have x-pack installed. What I'm trying to do is enable pipelines and monitoring for logstash through kibana.

The pipeline configuration in logstash.yml is working.

Unfortunately I can't get the monitoring feature to work:

2018-07-30T13:38:04.25+0200 [APP/PROC/WEB/0] OUT [2018-07-30T11:38:04,258][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://mynewuser:xxxxxx@https://myelasticsearch.cs.example.com:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'https://myelasticsearch.cs.example.com:9200/'"}

My question is, do I have to use the logstash_system user for the monitoring or is it ok to create a new native user? Weirdly the error persists even if the new user has the superuser role (login to kibana works fine).

Further question, if I need the logstash_system User, how would I create it? There seems to be no logstash_system-User on my Cluster (there is an "elastic"-User):

curl -XPUT _xpack/security/user/logstash_system/_enable'
{"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: only > existing users can be enabled;"}],"type":"validation_exception","reason":"Validation > Failed: 1: only existing users can be enabled;"},"status":400}

logstash.yml:

xpack.monitoring.enabled: "true"
xpack.monitoring.elasticsearch.url: "https://myelasticsearch.cs.example.com:9200"
xpack.monitoring.elasticsearch.username: "mynewuser"
xpack.monitoring.elasticsearch.password: "randompassword"

Thank you!

What versions of Elasticsearch and Logstash are you running?

I'm running 6.1.3, sorry for the missing information.

Did you copy this exactly from your cluster?
As best I can tell that error message has not existed in any recent version of X-Pack/Elasticsearch (I checked from 6.0.0 to 6.2.0)

What do you get for:

GET /_xpack/security/user/

and

GET /_xpack/
Did you copy this exactly from your cluster?

Yes (logstash and elasticsearch version 6.1.3)

GET /_xpack/security/user/

List of native users, comparable to the management/user section in kibana (no built-in users)

GET /_xpack/

{"build":{"hash":"9b1be50","date":"2018-01-26T19:20:38.715Z"},"license":{"uid":"f943086f-xxxx-xxxx-xxxx-b2b691dec9b7","type":"platinum","mode":"platinum","status":"active","expiry_date_in_millis":1535759999999},"features":{"graph":{"description":"Graph Data Exploration for the Elastic Stack","available":true,"enabled":true},"logstash":{"description":"Logstash management component for X-Pack","available":true,"enabled":true},"ml":{"description":"Machine Learning for the Elastic Stack","available":true,"enabled":false,"native_code_info":{"version":"N/A","build_hash":"N/A"}},"monitoring":{"description":"Monitoring for the Elastic Stack","available":true,"enabled":true},"security":{"description":"Security for the Elastic Stack","available":true,"enabled":true},"watcher":{"description":"Alerting, Notification and Automation for the Elastic Stack","available":true,"enabled":true}},"tagline":"You know, for X"}

I've worked out where that validation error is coming from.
Is your cluster running under Elastic Cloud Enterprise?

If so, unfortunately you can't use the logstash_system user, and will need to create your own custom user.

So, back to your original questions:

A native user will be fine. Just create logstash_monitoring user, and give it the logstash_system role.

I don't understand why that would be the case.
Are you sure the password is correct, and doesn't contain any unusual characters that might get incorrectly interpretted in the config file?

Thx for your comments Tim.

As it turns out there is a logstash_system user after all, the monitoring works when I use that one.

Sorry for the troubles.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.