K8s_node_name not found?

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hi guys,

I'm trying to follow the instructions here to setup a multi-az cluster: https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-advanced-node-scheduling.html and I've setup cluster.routing.allocation.awareness.attributes: k8s_node_name,zone in the config.

However Kibana fails to start up, and digging into the cluster allocation via curl -k -u elastic:T708a8w8PdGypG047UgW28Vo https://tenant-es-es-http.test-tenant-hxcwf:9200/_cluster/allocation/explain?pretty it seems like k8s_node_name is not set for the nodes:

    {
      "index" : ".kibana_1",
      "shard" : 0,
      "primary" : true,
      "current_state" : "unassigned",
      "unassigned_info" : {
        "reason" : "INDEX_CREATED",
        "at" : "2020-09-03T06:28:44.747Z",
        "last_allocation_status" : "no"
      },
      "can_allocate" : "no",
      "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
      "node_allocation_decisions" : [
        {
          "node_id" : "i-zJUAumRFGI8sWZcugI3g",
          "node_name" : "tenant-es-es-ap-southeast-1c-0",
          "transport_address" : "10.0.3.178:9300",
          "node_attributes" : {
            "ml.machine_memory" : "2147483648",
            "ml.max_open_jobs" : "20",
            "xpack.installed" : "true",
            "ml.enabled" : "true",
            "zone" : "ap-southeast-1c"
          },
          "node_decision" : "no",
          "weight_ranking" : 1,
          "deciders" : [
            {
              "decider" : "awareness",
              "decision" : "NO",
              "explanation" : "node does not contain the awareness attribute [k8s_node_name]; required attributes cluster setting [cluster.routing.allocation.awareness.attributes=k8s_node_name,zone]"
            }
          ]
        },
    ... two other nodes here

Is there a step I've missed? I haven't done anything explicit to set that k8s_node_name attribute as the docs have: "Note that by default ECK creates a k8s_node_name attribute with the name of the Kubernetes node running the Pod, and configures Elasticsearch to use this attribute".

Thanks!

2

This feature is not released yet. You were looking at the documentation for the master branch which is the version currently under development.

You can find the documentation for the current version by replacing master with current or the specific version of ECK you are running e.g.

https://www.elastic.co/guide/en/cloud-on-k8s/1.2/k8s-advanced-node-scheduling.html

1 Like