How to setup fluentd with elastic on cloud

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
3

What you are doing should work fine, but to make it better you'd probably want to:

  • mount the secret containing the elasticsearch password, instead of copying its value there:
name: FLUENT_ELASTICSEARCH_PASSWORD
valueFrom:
  secretKeyRef:
     name: data-es-es-elastic-user
     key: elastic
  • mount the TLS certificates from data-es-es-http-certs-public in the fluentd Pod, and configure it to use those certificates instead of setting FLUENT_ELASTICSEARCH_SSL_VERIFY : false

Note we're working on support for Beat in ECK. Filebeat allows you to ship all your Pod logs to Elasticsearch. Upcoming ECK release 1.2 will provide native support for filebeat, so ECK takes care of all the Beat->Elasticsearch configuration (url, user, password, TLS certificates, etc.).