ECK not working with ISTIO

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

I am testing the ECK with istio service mesh.

Used the documentation provided.

It worked fine when istio is installed without istio CNI

We are using istio 1.4.3

When istio CNI is installed, elastic components are not able to communicate with each other

{"type": "server", "timestamp": "2020-04-10T14:22:58,205Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "test-elastic", "node.name": "test-elastic-es-master-nodes-0", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [test-elastic-es-master-nodes-0, test-elastic-es-master-nodes-1, test-elastic-es-master-nodes-2] to bootstrap a cluster: have discovered [{test-elastic-es-master-nodes-0}{XN50bfmpTAO5yhIuwXH0Pw}{5aCYo0QIRzS7Oujah6YcEQ}{192.168.201.11}{192.168.201.11:9300}{m}{xpack.installed=true}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305, [::1]:9300, [::1]:9301, [::1]:9302, [::1]:9303, [::1]:9304, [::1]:9305, 192.168.106.77:9300, 192.168.239.150:9300] from hosts providers and [{test-elastic-es-master-nodes-0}{XN50bfmpTAO5yhIuwXH0Pw}{5aCYo0QIRzS7Oujah6YcEQ}{192.168.201.11}{192.168.201.11:9300}{m}{xpack.installed=true}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" }

{"type": "server", "timestamp": "2020-04-10T14:23:08,207Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "test-elastic", "node.name": "test-elastic-es-master-nodes-0", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [test-elastic-es-master-nodes-0, test-elastic-es-master-nodes-1, test-elastic-es-master-nodes-2] to bootstrap a cluster: have discovered [{test-elastic-es-master-nodes-0}{XN50bfmpTAO5yhIuwXH0Pw}{5aCYo0QIRzS7Oujah6YcEQ}{192.168.201.11}{192.168.201.11:9300}{m}{xpack.installed=true}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305, [::1]:9300, [::1]:9301, [::1]:9302, [::1]:9303, [::1]:9304, [::1]:9305, 192.168.106.77:9300, 192.168.239.150:9300] from hosts providers and [{test-elastic-es-master-nodes-0}{XN50bfmpTAO5yhIuwXH0Pw}{5aCYo0QIRzS7Oujah6YcEQ}{192.168.201.11}{192.168.201.11:9300}{m}{xpack.installed=true}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" }

2

It appears that the CNI plugin requires an additional annotation (traffic.sidecar.istio.io/includeInboundPorts: "*") to be added to the pod. The following manifest works with Istio 1.5.0 configured with the CNI plugin:

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elastic-istio
spec:
  version: 7.6.1
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
  - name: default
    count: 3
    podTemplate:
      metadata:
        annotations:
          traffic.sidecar.istio.io/includeInboundPorts: "*"
          traffic.sidecar.istio.io/excludeOutboundPorts: "9300"
          traffic.sidecar.istio.io/excludeInboundPorts: "9300"