How to communicate between ELS Nodes using Kubernetes cluster with namespace and SVC

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

I am trying to communicate between Elasticsearch nodes, so that the master node will be chosen in a voting. Unfortunately, nodes are not seeing eachother (discovery.seed_hosts), despite of fact, that these are chained using headless service. Moreove, pods are defined within the same namespace.

Service definition:

kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-scv
  namespace: elasticsearch-namespace
  labels:
    app: elasticsearch
spec:
  selector:
    app: elasticsearch
  clusterIP: None
  ports:
    - port: 9200
      name: rest
    - port: 9300
      name: inter-node

StatefulSet definition:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: es-cluster
  namespace: elasticsearch-namespace
spec:
  serviceName: elasticsearch
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
        resources:
            limits:
              cpu: 1000m
            requests:
              cpu: 100m
        ports:
        - containerPort: 9200
          name: rest
          protocol: TCP
        - containerPort: 9300
          name: inter-node
          protocol: TCP
        volumeMounts:
          - name: elasticsearch-persistent-storage
            mountPath: /usr/share/elasticsearch/data
        env:
          - name: cluster.name
            value: k8s-logs
          - name: node.name
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: discovery.seed_hosts
            value: "es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch"
          - name: cluster.initial_master_nodes
            value: "es-cluster-0,es-cluster-1,es-cluster-2"
          - name: node.max_local_storage_nodes
            value: "15"
          - name: ES_JAVA_OPTS
            value: "-Xms512m -Xmx512m"
          - name: node.max_local_storage_nodes
            value: "15"
      initContainers:
      - name: fix-permissions
        image: busybox
        command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
        securityContext:
          privileged: true
        volumeMounts:
        - name: elasticsearch-persistent-storage
          mountPath: /usr/share/elasticsearch/data
      - name: increase-vm-max-map
        image: busybox
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        securityContext:
          privileged: true
      - name: increase-fd-ulimit
        image: busybox
        command: ["sh", "-c", "ulimit -n 65536"]
        securityContext:
          privileged: true
      volumes:
        - name: elasticsearch-persistent-storage
          persistentVolumeClaim:
            claimName: elasticsearch-pvc

Namespace definition:

apiVersion: v1
kind: Namespace
metadata:
  name: elasticsearch-namespace
  labels:
    name: elasticsearch-namespace

@Edit

I have been trying to get the proper DNS using following commands:

k8 exec -it -n elasticsearch-namespace es-cluster-2 ping es-cluster-1.elasticsearch.elasticsearch-namespace.svc.es-cluster

k8 exec -it -n elasticsearch-namespace es-cluster-2 ping es-cluster-1.elasticsearch-scv.elasticsearch-namespace.svc.es-cluster

k8 exec -it -n elasticsearch-namespace es-cluster-2 ping es-cluster-1.elasticsearch.elasticsearch-scv.elasticsearch-namespace.svc.es-cluster

k8 exec -it -n elasticsearch-namespace es-cluster-2 ping es-cluster-1.elasticsearch.elasticsearch-namespace.svc.cluster.local

but I have found nothing at these addresses

@Edit2

CoreDNS output:

apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          upstream
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"Corefile":".:53 {\n    errors\n    health\n    kubernetes cluster.local in-addr.arpa ip6.arpa {\n      pods insecure\n      upstream\n      fallthrough in-addr.arpa ip6.arpa\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"eks.amazonaws.com/component":"coredns","k8s-app":"kube-dns"},"name":"coredns","namespace":"kube-system"}}
  creationTimestamp: "2020-06-10T09:38:23Z"
  labels:
    eks.amazonaws.com/component: coredns
    k8s-app: kube-dns
  name: coredns
  namespace: kube-system
  resourceVersion: "171"
  selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
  uid: 20d7c02d-bkgt-11ea-hf54-0240aa367d4c

/etc/resolv.conf

nameserver 172.20.0.10
search elasticsearch-namespace.svc.cluster.local svc.cluster.local cluster.local eu-central-1.compute.internal
options ndots:5

@Edit3

04:10 PM :- macbook @ ~/Desktop/projects $ k8 exec -it -n elasticsearch-namespace es-cluster-2 ping es-cluster-1.elasticsearch.elasticsearch-namespace.svc.cluster.local
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
ping: es-cluster-1.elasticsearch.elasticsearch-namespace.svc.cluster.local: Name or service not known
command terminated with exit code 2

Following URL does not work:
es-cluster-1.elasticsearch.elasticsearch-namespace.svc.cluster.local

2

Is there a reason for you to set these? ECK already handles it for you.