# Metricbeat is unable to send data to elastic search(011Attempting to reconnect to backoff(elasticsearch(http://x.x.x.x:9200)) with 441 reconnect attempt(s)) **URL:** \ **Category:** Kibana\ **Created:** [September 27, 2020, 6:55pm UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102 "2020-09-27T18:55:18Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 27, 2020, 6:55pm UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/1 "2020-09-27T18:55:19Z")
I have recently updated ES, Kibana, Logstash, All the services are running but the data is not getting collected. and UI is also loading. Please help me to resolve this. Kibana Log: {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["info","savedobjects-service"],"pid":18273,"message":"Starting saved objects migrations"} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["info","savedobjects-service"],"pid":18273,"message":"Creating index .kibana\_task\_manager\_3."} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["info","savedobjects-service"],"pid":18273,"message":"Creating index .kibana\_3."} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["warning","savedobjects-service"],"pid":18273,"message":"Unable to connect to Elasticsearch. Error: [resource\_already\_exists\_exception] index [.kibana\_task\_manager\_3/37\_qO60MRAOHhCGjCgz7RA] already exists, with { index\_uuid="37\_qO60MRAOHhCGjCgz7RA" & index=".kibana\_task\_manager\_3" }"} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["warning","savedobjects-service"],"pid":18273,"message":"Another Kibana instance appears to be migrating the index. Waiting for that migration to complete. If no other Kibana instance is attempting migrations, you can get past this message by deleting index .kibana\_task\_manager\_3 and restarting Kibana."} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["warning","savedobjects-service"],"pid":18273,"message":"Unable to connect to Elasticsearch. Error: [resource\_already\_exists\_exception] index [.kibana\_3/ej5RzANpSr-L86WUfe6PFA] already exists, with { index\_uuid="ej5RzANpSr-L86WUfe6PFA" & index=".kibana\_3" }"} {"type":"log","@timestamp":"2020-09-27T14:19:52Z","tags":["warning","savedobjects-service"],"pid":18273,"message":"Another Kibana instance appears to be migrating the index. Waiting for that migration to complete. If no other Kibana instance is attempting migrations, you can get past this message by deleting index .kibana\_3 and restarting Kibana."} {"type":"log","@timestamp":"2020-09-27T18:25:40Z","tags":["info","cli","config"],"pid":18273,"message":"Reloading logging configuration due to SIGHUP."} {"type":"log","@timestamp":"2020-09-27T18:25:40Z","tags":["info","cli","config"],"pid":18273,"message":"Reloaded logging configuration due to SIGHUP."} {"type":"log","@timestamp":"2020-09-27T18:25:40Z","tags":["info","plugins-system"],"pid":18273,"message":"Stopping all plugins."} ES Log: [2020-09-27T13:37:17,968][WARN][r.suppressed] [node-1] path: /.kibana\_task\_manager/\_count, params: {index=.kibana\_task\_manager} org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:551) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:309) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:582) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:393) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$0(AbstractSearchAsyncAction.java:223) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.action.search.AbstractSearchAsyncAction$2.doRun(AbstractSearchAsyncAction.java:288) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-7.9.2.jar:7.9.2] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.2.jar:7.9.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2020, 3:29am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/2 "2020-09-28T03:29:37Z")
You haven't provided anything that shows Metricbeat isn't sending data to Elasticsearch, so it's hard to say what the issue is. The warning you show shouldn't stop Metricbeat. ---
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 28, 2020, 4:11am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/3 "2020-09-28T04:11:09Z")
Here is the log. Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011ERROR#011pipeline/output.go:100#011Failed to connect to backoff(elasticsearch([http://x.x.x.x:9200](http://x.x.x.x:9200))): Get [http://x.x.x.x:9200](http://x.x.x.x:9200): net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011INFO#011pipeline/output.go:93#011Attempting to reconnect to backoff(elasticsearch([http://x.x.x.x:9200](http://x.x.x.x:9200))) with 441 reconnect attempt(s) Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011INFO#011[publisher]#011pipeline/retry.go:189#011retryer: send unwait-signal to consumer Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011INFO#011[publisher]#011pipeline/retry.go:191#011 done Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011INFO#011[publisher]#011pipeline/retry.go:166#011retryer: send wait signal to consumer Sep 27 03:47:29 li2111-133 metricbeat: 2020-09-27T03:47:29.476Z#011INFO#011[publisher]#011pipeline/retry.go:168#011 done ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2020, 4:13am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/4 "2020-09-28T04:13:34Z")
What is the output from the `_cat/health` API on Elasticsearch? ---
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 28, 2020, 4:24am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/5 "2020-09-28T04:24:32Z")
> [@warkolm](#): > > `_cat/health` 1601267052 04:24:12 elasticsearch red 1 1 308 308 0 3 64 1 - 82.1% ---
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 28, 2020, 4:30am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/6 "2020-09-28T04:30:31Z")
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending\_tasks max\_task\_wait\_time active\_shards\_percent 1601267635 04:33:55 elasticsearch yellow 1 1 366 366 0 0 9 0 - 97.6% ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2020, 4:48am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/7 "2020-09-28T04:48:22Z")
Ok that's good, it means that the cluster is recovering. Is Metricbeat sending now? ---
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 28, 2020, 4:52am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/8 "2020-09-28T04:52:12Z")
No. Not yet. but Port 9200 itself is not connecting from any other machines on which the metricbeat is installed(no firewall restrictions). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2020, 4:53am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/9 "2020-09-28T04:53:02Z")
Do you mean you cannot connect to port 9200 on Elasticsearch? ---
**Author:** ![NiranjanMurali](https://avatars.discourse-cdn.com/v4/letter/n/2acd7d/32.png) [@NiranjanMurali](https://discuss.elastic.co/u/NiranjanMurali)\ **Post date:** [September 28, 2020, 4:55am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/10 "2020-09-28T04:55:34Z")
Yes.,from remote machines(metricbeat) Curl output: curl: (7) Failed connect to 169.38.68.40:9200; Connection timed out ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2020, 4:56am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/11 "2020-09-28T04:56:25Z")
Can you run a curl to the Elasticsearch host from one of the machines? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 26, 2020, 4:56am UTC](https://discuss.elastic.co/t/metricbeat-is-unable-to-send-data-to-elastic-search-011attempting-to-reconnect-to-backoff-elasticsearch-http-x-x-x-x-9200-with-441-reconnect-attempt-s/250102/12 "2020-10-26T04:56:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.