How to config remote monitoring for Elasticsearch node in kubernetes

Elastic Stack Elasticsearch
1

Hi All,
I have a Elasticsearch node setup in kubernetes cluster with following configuration. Another ES node to store the monitoring data. But how to pass the http exporter config to yaml file ?

Both Elasticsearch Version: 6.8.0

"containers": [
  {
"name": "elasticsearch",
"image": "docker.elastic.co/elasticsearch/elasticsearch:6.8.0",
"ports": [
  {
    "name": "http",
    "containerPort": 9200,
    "protocol": "TCP"
  },
  {
    "name": "transport",
    "containerPort": 9300,
    "protocol": "TCP"
  }
],
"env": [
  {
    "name": "node.name",
    "valueFrom": {
      "fieldRef": {
        "apiVersion": "v1",
        "fieldPath": "metadata.name"
      }
    }
  },
  {
    "name": "discovery.zen.minimum_master_nodes",
    "value": "1"
  },
  {
    "name": "discovery.zen.ping.unicast.hosts",
    "value": "elasticsearch-master-headless"
  },
  {
    "name": "cluster.name",
    "value": "elasticsearch"
  },
  {
    "name": "network.host",
    "value": "0.0.0.0"
  },
  {
    "name": "ES_JAVA_OPTS",
    "value": "-Xmx1g -Xms1g"
  },
  {
    "name": "node.data",
    "value": "true"
  },
  {
    "name": "node.ingest",
    "value": "true"
  },
  {
    "name": "node.master",
    "value": "true"
  }
],
"resources": {
  "limits": {
    "cpu": "1500m",
    "memory": "3Gi"
  },
  "requests": {
    "cpu": "1",
    "memory": "3Gi"
  }
},
"volumeMounts": [
  {
    "name": "elasticsearch-master",
    "mountPath": "/usr/share/elasticsearch/data"
  },
  {
    "name": "default-token-6gzvx",
    "readOnly": true,
    "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
  }
],
"readinessProbe": {
  "exec": {
    "command": [
      "sh",
      "-c",
      "#!/usr/bin/env bash -e\n# If the node is starting up wait for the cluster to be ready (request params: 'wait_for_status=green&timeout=1s' )\n# Once it has started only check that the node itself is responding\nSTART_FILE=/tmp/.es_start_file\n\nhttp () {\n    local path=\"${1}\"\n    if [ -n \"${ELASTIC_USERNAME}\" ] && [ -n \"${ELASTIC_PASSWORD}\" ]; then\n      BASIC_AUTH=\"-u ${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}\"\n    else\n      BASIC_AUTH=''\n    fi\n    curl -XGET -s -k --fail ${BASIC_AUTH} http://127.0.0.1:9200${path}\n}\n\nif [ -f \"${START_FILE}\" ]; then\n    echo 'Elasticsearch is already running, lets check the node is healthy'\n    http \"/\"\nelse\n    echo 'Waiting for elasticsearch cluster to become cluster to be ready (request params: \"wait_for_status=green&timeout=1s\" )'\n    if http \"/_cluster/health?wait_for_status=green&timeout=1s\" ; then\n        touch ${START_FILE}\n        exit 0\n    else\n        echo 'Cluster is not yet ready (request params: \"wait_for_status=green&timeout=1s\" )'\n        exit 1\n    fi\nfi\n"
    ]
  },
  "initialDelaySeconds": 10,
  "timeoutSeconds": 5,
  "periodSeconds": 10,
  "successThreshold": 3,
  "failureThreshold": 3
},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent"
  }
]

All the elasticsearch.yml configs are passed as env variable.
Eg.
discovery.zen.minimum_master_nodes: 1

How to pass the value for remote monitoring as per the doc @ Collecting Monitoring Data

xpack.monitoring.exporters:
  id1:
    type: http
    host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
2

Added the below config and the remote monitoring started working.

      {
        "name": "xpack.monitoring.collection.enabled",
        "value": "true"
      },
      {
        "name": "xpack.monitoring.exporters.my_remote.type",
        "value": "http"
      },
      {
        "name": "xpack.monitoring.exporters.my_remote.host",
        "value": "http://<remote_es_ip>:9200"
      }
3

@tamilsweet We dropped the ball on getting you a timely answer. Apologies for that! Glad you got this working, though. Cheers. :slight_smile:

1 Like