I took over an elastic cluster with 3 nodes. All of the nodes have x-pack installed. What I'm trying to do is enable pipelines and monitoring for logstash through kibana.
The pipeline configuration in logstash.yml is working.
Unfortunately I can't get the monitoring feature to work:
2018-07-30T13:38:04.25+0200 [APP/PROC/WEB/0] OUT [2018-07-30T11:38:04,258][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://mynewuser:xxxxxx@https://myelasticsearch.cs.example.com:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'https://myelasticsearch.cs.example.com:9200/'"}
My question is, do I have to use the logstash_system user for the monitoring or is it ok to create a new native user? Weirdly the error persists even if the new user has the superuser role (login to kibana works fine).
Further question, if I need the logstash_system User, how would I create it? There seems to be no logstash_system-User on my Cluster (there is an "elastic"-User):
curl -XPUT _xpack/security/user/logstash_system/_enable'
{"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: only > existing users can be enabled;"}],"type":"validation_exception","reason":"Validation > Failed: 1: only existing users can be enabled;"},"status":400}
Did you copy this exactly from your cluster?
As best I can tell that error message has not existed in any recent version of X-Pack/Elasticsearch (I checked from 6.0.0 to 6.2.0)
List of native users, comparable to the management/user section in kibana (no built-in users)
GET /_xpack/
{"build":{"hash":"9b1be50","date":"2018-01-26T19:20:38.715Z"},"license":{"uid":"f943086f-xxxx-xxxx-xxxx-b2b691dec9b7","type":"platinum","mode":"platinum","status":"active","expiry_date_in_millis":1535759999999},"features":{"graph":{"description":"Graph Data Exploration for the Elastic Stack","available":true,"enabled":true},"logstash":{"description":"Logstash management component for X-Pack","available":true,"enabled":true},"ml":{"description":"Machine Learning for the Elastic Stack","available":true,"enabled":false,"native_code_info":{"version":"N/A","build_hash":"N/A"}},"monitoring":{"description":"Monitoring for the Elastic Stack","available":true,"enabled":true},"security":{"description":"Security for the Elastic Stack","available":true,"enabled":true},"watcher":{"description":"Alerting, Notification and Automation for the Elastic Stack","available":true,"enabled":true}},"tagline":"You know, for X"}
I've worked out where that validation error is coming from.
Is your cluster running under Elastic Cloud Enterprise?
If so, unfortunately you can't use the logstash_system user, and will need to create your own custom user.
So, back to your original questions:
A native user will be fine. Just create logstash_monitoring user, and give it the logstash_system role.
I don't understand why that would be the case.
Are you sure the password is correct, and doesn't contain any unusual characters that might get incorrectly interpretted in the config file?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.