Filebeat to ECK on GKE

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

I built a cluster on GKE with ECK operator and am trying to send logs from an on premises Filebeat installation to the cloud.

Elasticsearch has LoadBlancer IP. I specified certificate, password and necessary things, but I couldn't make it work. Is there a tutorial?

2

Hi @Zero_Day, thanks for your message.

While not reflecting an exactly same setup, we have a sample that contains filebeat configuration - https://github.com/elastic/cloud-on-k8s/blob/master/config/recipes/beats/2_filebeat-kubernetes.yaml.

Are there any specific errors, logs, configs, that you could share? Is the Elasticsearch reachable (e.g. with curl) from the host running filebeat through that LoadBalancerIP?

3

Hi,

First of all thanks for the quick reply. Error I am getting related to x509 cert. I got the cert from elasticsearch-ca-internal. Still same error.

4

I think the issue you are hitting is related to SANs on the certificate. Currently, we don't populate the generated certificate with LoadBalancer IP as IP SAN. This means that when you use IP address to connect to ES, TLS connection can't be established due to that IP not being present on the cert.

To resolve it right now, you can change your ES spec to include the following:

spec:
...
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        subjectAltNames:
        - ip: IP_ADDRESS

where IP_ADDRESS is the external (LoadBalancer) IP that you are using to connect to ES.

Enhancement that would make this seamless (ie. would not require the tls config section to be added by the user) is tracked at https://github.com/elastic/cloud-on-k8s/issues/910.