Unassigned shard after Kibana 4 joins cluster

Elastic Stack Kibana
1

Hi all,

I am pretty new to ELK.

I am trying at this stage just to build ELK on a single node and I have Elasticsearch working fine until Kibana 4 adds its index in.

At this point the cluster goes yellow and I end up with one unassigned shard:

[root@centos-66-x64 ~]# curl localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "es01",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 6,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 1,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 85.71428571428571
}

Indices:

[root@centos-66-x64 ~]# curl localhost:9200/_cat/indices
green  open logstash-2016.07.27 1 0  28 0  34.2kb  34.2kb 
green  open logstash-2016.08.06 1 0 379 0 247.8kb 247.8kb 
green  open logstash-2016.07.26 1 0 361 0 245.6kb 245.6kb 
green  open logstash-2016.01.24 1 0 114 0 121.7kb 121.7kb 
yellow open .kibana             1 1   1 0   3.1kb   3.1kb 
green  open logstash-2016.01.23 1 0 591 0 403.5kb 403.5kb

My Kibana config:

[root@centos-66-x64 ~]# cat /opt/kibana-4.3.1-linux-x64/config/kibana.yml 
elasticsearch_url: http://localhost:9200
host: 0.0.0.0
pid.file: /var/run/kibana4/kibana4.pid
port: 5601

Let me know if other info is required. I'm hitting the 5000 character limit when I add log files.

2

Because KB creates an index, called .kibana, and it has one replica.
Given you have one node you cannot assign the primary and the replica to the same node, so it stays unassigned.

3

Thanks for your help. So do you mean there's really no clean way to run ELK all on a single node?

4

Define clean.

If you want a green cluster, just set replicas to 0 for all indices.

5

I see. So this works, as you say:

[root@centos-66-x64 ~]# curl 'localhost:9200/.kibana/_settings?pretty'
{
  ".kibana" : {
    "settings" : {
      "index" : {
        "creation_date" : "1453603521524",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "9eQozCRmQNimeB07HImGSQ",
        "version" : {
          "created" : "2010199"
        }
      }
    }
  }
}

Change settings:

[root@centos-66-x64 ~]# curl -XPUT 'localhost:9200/.kibana/_settings' -d '{"index":{"number_of_replicas":0}}'
[root@centos-66-x64 ~]# curl 'localhost:9200/.kibana/_settings?pretty'
{
  ".kibana" : {
    "settings" : {
      "index" : {
        "creation_date" : "1453603521524",
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "uuid" : "9eQozCRmQNimeB07HImGSQ",
        "version" : {
          "created" : "2010199"
        }
      }
    }
  }
}
[root@centos-66-x64 ~]# curl localhost:9200/_cluster/health?pretty 
{
  "cluster_name" : "es01",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 6,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
6

It looks like there is no way to tell Kibana 4 not to set replicas on its index in the first place however?

7

The default is to have one replica, irrespective of the index type.

It's not really worth worrying about unless you are constantly recreating the .kibana index. And if you are, just use curl -XPUT localhost:9200/*/_settings -d '{ "index" : { "number_of_replicas" : 0 } }' to make things green.

8

Well it's a bit of an issue as people will be using config management systems like Puppet, as I am. I'm trying to spin all of this up on one node for the purpose of serverspec testing in a CI pipeline. I can do it using a Puppet exec, but it's ugly.

9

Then does it really matter if the cluster is yellow?

10

Well it matters in so far as one of my acceptance criteria is that the code builds a green cluster. :slight_smile: Anyhow, thanks very much for your help.