Connect to Elasticsearch with TransportClient

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hello,

Even if java TransportClient is deprecated, I need to connect on port 9300 to an elasticsearch cluster deployed with ECK operator.

Devops team has worked to open port 9300 on quickstart-es-http service so that it looks like this :

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2020-01-17T10:33:56Z"
  labels:
   common.k8s.elastic.co/type: elasticsearch
   elasticsearch.k8s.elastic.co/cluster-name: quickstart
 name: quickstart-es-http
 namespace: default
 ownerReferences:
 - apiVersion: elasticsearch.k8s.elastic.co/v1
   blockOwnerDeletion: true
   controller: true
   kind: Elasticsearch
    name: quickstart
    uid: dd235882-3914-11ea-b5af-42010a8401b1
  resourceVersion: "209164059"
  selfLink: /api/v1/namespaces/default/services/quickstart-es-http
  uid: dd27e974-3914-11ea-b5af-42010a8401b1
spec:
  clusterIP: 10.47.249.165
  ports:
  - name: http
    port: 9200
    protocol: TCP
    targetPort: 9200
  - name: inc
    port: 9300
    protocol: TCP
    targetPort: 9300
  selector:
    common.k8s.elastic.co/type: elasticsearch
    elasticsearch.k8s.elastic.co/cluster-name: quickstart
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

... but I still cannot connect to the cluster with the TransportClient (famous NoNodeAvailableException) whereas I have no problem to connect with the java high level REST client.

Any tips to fix this issue?

2

Hello,

Can you give more details about your transport client setup? Which URL does it use to connect to the nodes, which certificates, etc.
The service looks correct to me, but we would need to spend some time trying to reproduce with the transport client, which is deprecated.

3

I am trying to connect an opensource service called ARLAS-server which instantiate TransportClient here and depends on elasticsearch 7.3.2.

I think it should work properly without enabling ssl (i.e. no XPackTransportClient) and without sniffing... but it cannot connect. To fix this, I tried different client setup available with ARLAS-server, but none of them is working.

I deploy ARLAS-server container in the same GKE cluster and use the following config :

  • transport adress : quickstart-es-http:9300
  • cluster.name : quickstart
4

I'm not familiar at all with ARLAS-server. A few things that may help you:

  • Are you able to reach the Elasticsearch port from your ARLAS-server container? You could try with something like nc -vz quickstart-es-http 9300.
  • According to that piece of code it looks like you can use SSL. Which is enabled by default. To verify the Elasticsearch server certificate you may need to provide the ca.crt stored in the <cluster-name>-es-transport-certs-public Kubernetes secret.
5

ARLAS-server container is able to reach Elasticsearch cluster on transport port (9300), that was one of the first things I checked when facing the NoNodeAvailableException.

We have switched off TLS on the cluster to avoid additional connection problem, but maybe it's only switched off for http (9200) and not for transport (9300). By reading this, it looks like transport layer is not configurable with ECK, right?

By the way, it looks like the cluster does not store the certificate you have mentioned :

% kubectl get secret | grep es-transport
%
% kubectl get secret | grep es-http
%

Maybe it's the root cause of my issue, but I cannot find documentation to fix it.

6

By reading this, it looks like transport layer is not configurable with ECK, right?

That's correct.

By the way, it looks like the cluster does not store the certificate you have mentioned

Hm this is weird, can you double check you are in the correct namespace? Do you see any ECK-managed secret at all?

7

If the transport layer is not configurable with ECK, does that mean that remote clusters cannot be used with ECK?
Reminding that to connect to a remote cluster, one needs to use the transport (9300) port, not http port (9200).
Any workarounds?

8

Please see the remote cluster documentation: https://www.elastic.co/guide/en/cloud-on-k8s/1.1/k8s-remote-clusters.html