Connecting Filebeat to and ECK from outside the K8s with ingress

Good morning.

From a Linux server out of the K8s where my ES is running, Im trying to setup the filebeat so it connects to the Service using the ingress.

If from the linux where I run the filebeat I run a curl to check the connectivity , it works:

Now Im trying to set up the filebeat.yaml with this settings:

  # Array of hosts to connect to.
  hosts: [""]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "${FB_USER}"
  password: "${FB_PWD}"
  ssl.verification_mode: "none"

Once I run the 'filebeat setup -e' I get the following error.

{"log.level":"error","@timestamp":"2023-06-08T12:54:58.624+0200","log.logger":"esclientleg","log.origin":{"":"eslegclient/connection.go","file.line":235},"message":"error connecting to Elasticsearch at Get "": dial tcp X.X.X.X:9200: connect: no route to host","":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-06-08T12:54:58.624+0200","log.origin":{"":"instance/beat.go","file.line":1071},"message":"Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at Get "": dial tcp X.X.X.X:9200: connect: no route to host]","":"filebeat","ecs.version":"1.6.0"}

Said that. As my curl really resolves an IP:9200 without addind manually the 9200 as it is included in the ingress. I've got the feeling that the error Filebeat is having is about translating the output.elasticsearch.hosts to -> IP:9200:9200

Does what Im saying makes any sense?

What do you think it could be the problem here? As I say I dont think there is a problem of firewalls or passwords as the curl works and the error message talks about what IMHO is a wrong host URL PATH.

Thank you in advance

Hi all,

I think that it depends of your CNI on kubernetes. In fact if you masquerade the output from your cluster to the external you must ensure that a worker could join the ES instance if not it explains why you have "no route to host". The pod try to connect to ES instance and use the "routing" method used internally on your cluster. Here I use cilium as CN and permit the usage of the local routing table of the nodes for routing trafic intern (pod) -> extern, the CNI cilium masquerade from each node traffic.

I think it's just that. If the node cannot join the ES instance you could maybe define an http proxy (which is joinable from the nodes) and define the right configuration to permit filebeat to go outside through the proxy.


1 Like

Thank you ccaillet.

I've finally fixed it by adding this line to my ingress.yaml "0"

kind: Ingress
  name: kibana
  namespace: monitoring
  annotations: / "0" HTTPS
  ingressClassName: nginx
    - host: kibanaeck.XXXXXXXXXcom
          - path: /
            pathType: Prefix
                name: kibana-kb-http
                  name: https

Thank you very much again :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.