Ingress for ElasticSearch using ECK

[sebastianlindblom]

I'm having trouble setting up a working ingress for ElasticSearch on kubernetes using the ECK API.

My elastic.yaml looks like this: (ingress.yaml is included at the bottom)

apiVersion: elasticsearch.k8s.elastic.co/v1beta1
kind: Elasticsearch
metadata:
  name: elasticsearch-test
  namespace: elastic-system
spec:
  version: 7.4.0
  http:
    service:
      spec:
        type: ClusterIP
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
  - name: master
    count: 1
    nodeSelector:
      component: elasticsearch
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-master
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: multik8s-nfs-storage
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: multik8s-nfs-storage
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
'''
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
  name: elasticsearch
  namespace: elastic-system
spec:
  tls:
    - hosts:
      - elasticsearch.foo.bar
      secretName: tls-secret
  rules:
    - host: elasticsearch.foo.bar
      http:
        paths:
          - path: /
            backend:
              serviceName: elasticsearch-test-es-http
              servicePort: 9200

And my kibana.yaml looks like this:

apiVersion: kibana.k8s.elastic.co/v1beta1
kind: Kibana
metadata:
  name: kibana-test
  namespace: elastic-system
spec:
  version: 7.4.0
  #http:
  #  tls:
  #    certificate:
  #      secretName: tls-secret-test
  http:
    service:
      spec:
        type: ClusterIP
    tls:
      selfSignedCertificate:
        disabled: true
  count: 1
  elasticsearchRef:
    name: elasticsearch-test
'''
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
  name: kibana
  namespace: elastic-system
spec:
  tls:
    - hosts:
      - kibana.foo.bar
      secretName: tls-secret
  rules:
    - host: kibana.foo.bar
      http:
        paths:
          - path: /
            backend:
              serviceName: kibana-test-kb-http
              servicePort: 5601

First of all, I do have my own signed tls cert i want to use in the ingress.
The wierd thing is that the ingress for kibana works straight out the box, without any problems. The ingress for elasticsearch only works while I'm on the same network as the k8s cluster. Not outside of it.

curl -u "elastic:$PASSWORD" -k "https://elasticsearch.foo.bar"
curl: (7) Failed to connect to elasticsearch.foo.bar port 443: Connection refused

On the same network I'm getting

curl -u "elastic:$PASSWORD" -k "https://elasticsearch.foo.bar"
{
  "name" : "elasticsearch-test-es-master-0",
  "cluster_name" : "elasticsearch-test",
  "cluster_uuid" : "ulfFb-tjT8KplEBPSglo6w",
  "version" : ...
}

I've experimented some by setting

tls:
  selfSignedCertificate:
    subjectAltNames:
      - dns: elasticsearch.foo.bar

and

tls:
  certificate:
    secretName: tls-secret-test

without success... But I'm guessing thats used for internal traffic, i.e between kibana and elasticsearch?

I'm not really sure what I'm doing wrong since it's working with Kibana but not ElasticSearch...

P.s both kibana and elastics healths are green:
i.e

NAME                         HEALTH   NODES   VERSION   PHASE   AGE
elasticsearch-test   green    1       7.4.0     Ready   1d
NAME                  HEALTH   NODES   VERSION   AGE
kibana-test   green    1       7.4.0     1d

[sebastianlindblom]

Noticed now that maybe this question is misplaced and should be placed in the ECK forum instead.