# Metricbeats failed to connect Elasticsearch with X-Pack enabled **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [July 15, 2019, 10:23am UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498 "2019-07-15T10:23:46Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\ **Post date:** [July 15, 2019, 10:23am UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/1 "2019-07-15T10:23:46Z")
I setup an Elasticsearch 7.2.0 with the following `elasticsearch.yml` config file, , using docker, ``` cluster.name: "docker-cluster" network.host: 0.0.0.0 discovery.type: single-node xpack.license.self_generated.type: basic xpack.security.enabled: true xpack.monitoring.collection.enabled: true ``` then I set password for built-in users using: ``` bin/elasticsearch-setup-passwords auto --batch ``` Also, in another (external) server, I setup Metricbeats 7.2.0 with the following config file again by docker, ``` output.elasticsearch: hosts: ["http://my_elasticsearch_server_ip:9200"] username: elastic password: RANDOM_PASS_GENERATED_FOR_ELASTIC_USER ``` but Metricbeats fails to connect Elasticsearch server, its log says: > metricbeat | 2019-07-15T10:08:11.144Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://my\_elasticsearch\_server\_ip:9200)): 401 Unauthorized: {"error":{"root\_cause":[{"type":"security\_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security\_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401} I should mention that, Metricbeats successfully connects to elastic when I set `xpack.security.enabled: false`. ---
**Author:** ![Michal\_Pristas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michal_pristas/32/46639_2.png) [@Michal\_Pristas](https://discuss.elastic.co/u/Michal_Pristas)\ **Post date:** [July 15, 2019, 11:41am UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/2 "2019-07-15T11:41:32Z")
> [@alza](#): > > failed to authenticate Does user have privileges to publish events? Are you able to log in using provided user:password combination? Do you see anything useful in ES logs? ---
**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\ **Post date:** [July 15, 2019, 11:54am UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/3 "2019-07-15T11:54:00Z")
> [@Michal\_Pristas](#): > > Does user have privileges to publish events? I assumed elastic user have that access. How can I check that? I can log in using "elastic" user and that password in Kibana, I can also get the result of: curl -X GET -u elastic:RANDOM\_PASS\_GENERATED\_FOR\_ELASTIC\_USER "http://my\_elasticsearch\_server\_ip:9200/\_cluster/health" log of ES: > elasticsearch\_1 | {"type": "server", "timestamp": "2019-07-15T11:53:29,411+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "docker-cluster", "node.name": "xxxxxx", "cluster.uuid": "xxxxxx", "node.id": "-xxxxxx", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } ---
**Author:** ![Michal\_Pristas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michal_pristas/32/46639_2.png) [@Michal\_Pristas](https://discuss.elastic.co/u/Michal_Pristas)\ **Post date:** [July 15, 2019, 12:26pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/4 "2019-07-15T12:26:39Z")
> [@alza](#): > > Authentication of [elastic] was terminated by realm can you check `security` index is there. if not please run `setup-passwords` again and configure metric beat with new password. Do you also installed x-pack on Kibana and provided same credentials there? Have you restarted metric beat after changing credentials? ---
**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\ **Post date:** [July 15, 2019, 12:49pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/5 "2019-07-15T12:49:04Z")
> [@Michal\_Pristas](#): > > can you check `security` index is there. Sorry I am a bit newb, just started elastic, what do you mean by this? Should I generate a specific password for Metricbeats or I can use elastic user and passwords in Metricbeats client? I setup ES, Kibana and Logstash using docker-compose, and set the passwords generated by `bin/elasticsearch-setup-passwords auto --batch` for Kibana and Logstash in their config files like: ``` elasticsearch.username: kibana elasticsearch.password: RANDOM_PASS_GENERATED_FOR_KIBANA ``` Yes, I have restarted Metricbeats several times so far. ---
**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\ **Post date:** [July 15, 2019, 12:55pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/6 "2019-07-15T12:55:40Z")
For the security index you can go into Kibana -\> Management (Gear Icon) -\> Index Management and then click on the Include System Indices in the upper right hand corner. That should list some extra indices and one should be .security. ---
**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\ **Post date:** [July 15, 2019, 1:05pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/7 "2019-07-15T13:05:32Z")
Thanks Michal and Ryan, my issue solved! Actually, the main problem was in docker-compose that I used for Metricbeats, it binds a config file to Metricbeats container, which I was changing elastic password there, but it also get elastic password from env variable and set it using docker-compose command: ``` command: metricbeat -e -system.hostfs=/hostfs -E output.elasticsearch.username=elastic -E output.elasticsearch.password=${ES_PASSWORD} -strict.perms=false ``` this command overwrites the password that I set in config file, I just removed -E from the above command and the issue resolved! ---
**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\ **Post date:** [July 15, 2019, 1:09pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/8 "2019-07-15T13:09:47Z")
You're welcome! I don't know everything about Elastic but try to help out in the forums when I can. Good luck on your future endeavors with Elastic. It can be tough for sure but its also a pretty powerful tool once you get used to it. ---
**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\ **Post date:** [July 15, 2019, 1:12pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/9 "2019-07-15T13:12:44Z")
Thanks! That is very good. Yes, Elastic is really great! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 12, 2019, 1:12pm UTC](https://discuss.elastic.co/t/metricbeats-failed-to-connect-elasticsearch-with-x-pack-enabled/190498/10 "2019-08-12T13:12:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.