# Unable to connect to Kibana

**URL:** <https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974>\
**Category:** Kibana\
**Created:** [August 6, 2020, 7:27am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974 "2020-08-06T07:27:35Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 7:27am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/1 "2020-08-06T07:27:36Z")

</div>

Hello,

I have configured ELK with a nginx in front for HTTPS and authentication. This setup worked perfectly until this morning (I stopped the server for days and restarted it).

WHen I try to connect to kibana with Firefox I have the following message "Firefox can’t establish a connection to the server at 192.168.100.100"

I can ping my server and the port 443 is open and listening.

On the server I can curl my elasticsearch "curl [http://localhost:9200](http://localhost:9200)"

I looked in Kibana and nginx logs but I have no specific error.

DIid someone already have similar issues?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 6, 2020, 7:28am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/2 "2020-08-06T07:28:47Z")

</div>

Why not use [free Security](https://www.elastic.co/blog/security-for-elasticsearch-is-now-free) to do this?

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 7:54am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/3 "2020-08-06T07:54:50Z")

</div>

If I am not mistaken, when I implemented this setup this functionality was not part of the free license.  
Before I stopped and restarted the server, this setup worked perfectly and I don't figure out the issue I have now.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 6, 2020, 10:05am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/4 "2020-08-06T10:05:35Z")

</div>

We use a very similar setup.

First, make sure the host where Nginx is running can connect to Kibana on the correct port. You can try something like

```
$ nc -v -z my_kibana_ip 5601
my.kibana.local [10.0.0.1] 5601 (?) open

```

or

```
$ curl -s -I http://my_kibaba_ip:5601
HTTP/1.1 302 Found
location: /spaces/enter
kbn-name: my.kibana.local
kbn-license-sig: d865465067a482e8b79266a0cacbbf52067452bb2d7eda73c659993b0a054995
kbn-xpack-sig: ef42adb69cd0ad9c36eb0323e8819b1d
cache-control: private, no-cache, no-store, must-revalidate
content-length: 0
Date: Thu, 06 Aug 2020 10:02:12 GMT
Connection: keep-alive

```

If that works, next check your Nginx config. You should definitely see some errors in the logs if Nginx can not reach one of the backends.

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 12:50pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/5 "2020-08-06T12:50:19Z")

</div>

Yes it works. Nginx can connect to Kibana.

I thought it was a network issue but apparently it is not because when I look at the elasticsearch log I find the below error:

```auto
[2020-08-06T12:37:10,303][WARN][r.suppressed] [miXbRBA] path: /.kibana_task_manager/_doc/_search, params: {ignore_unavailable=true, index=.kibana_task_manager, type=_doc}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed

[...]

[2020-08-06T12:45:42,739][INFO][o.e.c.r.a.DiskThresholdMonitor] [miXbRBA] low disk watermark [85%] exceeded on [miXbRBAXTkiAo4KZhbFevA][miXbRBA][/var/lib/elasticsearch/nodes/0] free: 1.3gb[13.3%], replicas will not be assigned to this node

```

So apparently it is due to low disk space but I am not sure how I can bypass the alert or move older data manually.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 6, 2020, 1:17pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/6 "2020-08-06T13:17:43Z")

</div>

I have been in that situation a few times as well.

The quickest and easiest solution is deleting some indices if you can afford to do that (you will loose data).

You could possibly also momentarily decrease the [replication factor](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html) if that is set high. It will decrease resilience but might be acceptable for some time.

Other less easy approaches would be to add ES nodes to the cluster to add total capacity. The cluster will balance storage over time and recover from your current situation.

Or you can add a [snapshot repo](https://www.elastic.co/guide/en/elasticsearch/plugins/master/repository.html) and backup data if you can't afford to loose any of the data permanently.

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 1:58pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/7 "2020-08-06T13:58:06Z")

</div>

What is the proper procedure to safely delete old indices ?

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 2:12pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/8 "2020-08-06T14:12:17Z")

</div>

I removed old indices and disk spaces is now free at 62%. But I still cannot access the kibana UI.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 6, 2020, 2:20pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/9 "2020-08-06T14:20:30Z")

</div>

Is the ES cluster health green?

This should show if there are any problems still with ES

```auto
curl -XGET es_host:9200/_cluster/allocation/explain?pretty

```

Check for `explanation` in the output.

Kibana can take a bit of time to start. Kibana logs are not always that clear I find as to what is the problem, but there should definitely be something in the logs if Kibana can't start.

Did you try `netcat` or something similar like `lsof`

```auto
$ nc -v -z my_kibana_ip 5601
my.kibana.local [10.0.0.1] 5601 (?) open

```

Is Kibana listening on its port or not?

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 6, 2020, 2:45pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/10 "2020-08-06T14:45:59Z")

</div>

Kibana start correctly but with some info logs:

```auto
 kibana.service - Kibana
     Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2020-08-06 14:06:34 UTC; 34min ago
   Main PID: 2876 (node)
      Tasks: 11 (limit: 9452)
     Memory: 254.9M
     CGroup: /system.slice/kibana.service
             └─2876 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml

Aug 06 14:06:46 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:46Z","tags":["status","plugin:security@6.8.9","info"],"pid":2876,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yel
low","prevMsg":"Waiting for Elasticsearch"}
Aug 06 14:06:46 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:46Z","tags":["status","plugin:maps@6.8.9","info"],"pid":2876,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow"
,"prevMsg":"Waiting for Elasticsearch"}
Aug 06 14:06:46 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:46Z","tags":["reporting","warning"],"pid":2876,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from fail
ing on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
Aug 06 14:06:47 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:47Z","tags":["status","plugin:reporting@6.8.9","info"],"pid":2876,"state":"green","message":"Status changed from uninitialized to green - Ready","prevSta
te":"uninitialized","prevMsg":"uninitialized"}
Aug 06 14:06:47 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:47Z","tags":["license","info","xpack"],"pid":2876,"message":"Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic |
status: active"}
Aug 06 14:06:47 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:47Z","tags":["info","migrations"],"pid":2876,"message":"Creating index .kibana_1."}
Aug 06 14:06:48 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:48Z","tags":["info","migrations"],"pid":2876,"message":"Pointing alias .kibana to .kibana_1."}
Aug 06 14:06:48 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:48Z","tags":["info","migrations"],"pid":2876,"message":"Finished in 446ms."}
Aug 06 14:06:48 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:48Z","tags":["listening","info"],"pid":2876,"message":"Server running at http://192.168.100.100:5601"}
Aug 06 14:06:48 KIBANA-srv kibana[2876]: {"type":"log","@timestamp":"2020-08-06T14:06:48Z","tags":["status","plugin:spaces@6.8.9","info"],"pid":2876,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yello
w","prevMsg":"Waiting for Elasticsearch"}

```

With the following command

> curl -XGET es\_host:9200/\_cluster/allocation/explain?pretty

I have this output:

```auto
{
  "index" : "winlogbeat-2020.08.06",
  "shard" : 3,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2020-08-06T14:35:54.238Z",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "miXbRBAXTkiAo4KZhbFevA",
      "node_name" : "miXbRBA",
      "transport_address" : "127.0.0.1:9300",
      "node_attributes" : {
        "ml.machine_memory" : "8349188096",
        "xpack.installed" : "true",
        "ml.max_open_jobs" : "20",
        "ml.enabled" : "true"
      },
      "node_decision" : "no",
      "weight_ranking" : 1,
      "deciders" : [
        {
          "decider" : "enable",
          "decision" : "NO",
          "explanation" : "replica allocations are forbidden due to cluster setting [cluster.routing.allocation.enable=primaries]"
        },
        {
          "decider" : "same_shard",
          "decision" : "NO",
          "explanation" : "the shard cannot be allocated to the same node on which a copy of the shard already exists [[winlogbeat-2020.08.06][3], node[miXbRBAXTkiAo4KZhbFevA], [P], s[STARTED], a[id=LveDU1oJTzWRySSK59yGyQ]]"
        }
      ]
    }
  ]
}

```

I tried to delete this index but I still have the same message.

For further information:

> curl localhost:9200/\_cat/indices?v

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana_1 dTDKefezRzaLXbuWj9qrig 1 0 2 0 7.6kb 7.6kb
green open .kibana_task_manager LsDYxnk6S0KpSTtmnuzv6A 1 0 2 0 12.6kb 12.6kb
yellow open winlogbeat-2020.08.06 GGu_OxJGT3C5DuXiIlYsdA 5 1 146 0 1.9mb 1.9mb

```

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 6, 2020, 3:07pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/11 "2020-08-06T15:07:29Z")

</div>

For `replica allocations are forbidden due to cluster setting [cluster.routing.allocation.enable=primaries]`

you should do

```auto
curl -X PUT "es_host:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": null
  }
}
'

```

That might solve the rest of the issues as well, not sure if you will have to initiate shard allocation with some sort of retry. I have had to do that in the past but it is not always needed.

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 7, 2020, 6:13am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/12 "2020-08-07T06:13:42Z")

</div>

@A_B Thanks for your help in this.

I have now the following error:

> "allocate\_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",

Is this the reason you said I have to initiate shard allocation ?

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 7, 2020, 7:14am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/13 "2020-08-07T07:14:49Z")

</div>

> "cannot allocate because allocation is not permitted to any of the nodes"

That could be a few things. The ones I have experienced myself would be

1. Not enough free space. Nodes will not be allowed to create new shards if they are over their _low water mark_
2. You have disabled shard allocation. Should not be that anymore.
3. We use \_ Cluster-level shard allocation\_ configurations that could cause it

I know it is a bit to read but your answer should be in this [document](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html).

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 7, 2020, 12:57pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/14 "2020-08-07T12:57:57Z")

</div>

Hi @pchar,

I should have thought about this right away...

Check the cluster status

```
GET _cluster/settings

```

Sharing it here could help identify possible reasons for your current problem as well 🙂

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 7, 2020, 1:05pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/15 "2020-08-07T13:05:47Z")

</div>

Please see the results below:

> {"persistent":{},"transient":{"cluster":{"routing":{"allocation":{"disk":{"watermark":{"low":"90%"}},"enable":"all"}}}}}

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 7, 2020, 1:17pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/16 "2020-08-07T13:17:19Z")

</div>

That looks like what I would expect it to. Is Kibana still not starting @pchar ?

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 7, 2020, 1:32pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/17 "2020-08-07T13:32:14Z")

</div>

I am not sure of the reason but the issue changed. Kibana now starts I just received "502 bad gateway" from nginx. But nginx is on the same server and when i tried on the server it works:

> curl -s -I [http://192.168.100.100:5601](http://192.168.100.100:5601)  
> HTTP/1.1 302 Found  
> location: /app/kibana  
> kbn-name: kibana  
> kbn-xpack-sig: 5c37b7e6745e03f5023340da2bd310b6  
> content-type: text/html; charset=utf-8  
> cache-control: no-cache  
> content-length: 0  
> connection: close  
> Date: Fri, 07 Aug 2020 13:27:49 GMT

At least now the issue as evolved and looks not linked to Kibana anymore...

---

<div class="post-metadata">

**Author:** ![pchar](https://avatars.discourse-cdn.com/v4/letter/p/7cd45c/32.png) [@pchar](https://discuss.elastic.co/u/pchar)\
**Post date:** [August 7, 2020, 1:48pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/18 "2020-08-07T13:48:17Z")

</div>

I had to make a small change in one nginx config file and now Kibana is working again.

Thanks @A_B for your support and your useful inputs.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2020, 1:48pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana/243974/19 "2020-09-04T13:48:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.