Filebeat running on Kubernetes connected to Elasticsearch using SSL

I would like to use filebeat on Kubernetes and connect it to a secured Elasticsearch server, using certificates. I did a search to find how to do this but only found an un-answered related question.
After some test I got it, so I will share my findings to help anybody else.
These are the steps:

  1. Follow the original documentation to download the yaml file.

  2. As shown on the documentation, update this file with the correct details about the Elasticsearch server: hostname, port, user and password

  3. Get a copy of the certificate to be used for the connection. In my example, I am using the "elasticsearch-ca.pem" file created when securing the server.

  4. Create a Kubernetes secret using the command:
    kubectl create secret generic elastic-ca --from-file=elasticsearch-ca.pem

  5. Edit the filebeat-kubernetes.yaml file as follows:
    At the beginning of the file, update the "output.elasticsearch" section content as follows (only the last 2 lines are added from the original content):

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}
      protocol: "https"
      ssl.certificate_authorities:  ["/etc/filebeat_ssl/elasticsearch-ca.pem"]
  1. Under the "volumeMounts" section, add the last 3 lines as follows:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: varlog
          mountPath: /var/log
          readOnly: true
        - name: ssl
          mountPath: /etc/filebeat_ssl
          readOnly: true
  1. Under the "volumes" section, add the last 3 lines as follows:
      volumes:
      - name: config
        configMap:
          defaultMode: 0640
          name: filebeat-config
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: varlog
        hostPath:
          path: /var/log
      # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
      - name: data
        hostPath:
          # When filebeat runs as non-root user, this directory needs to be writable by group (g+w).
          path: /var/lib/filebeat-data
          type: DirectoryOrCreate
      - name: ssl
        secret:
          secretName: elastic-ca
  1. You can now deploy the updated yaml file with the command:
    kubectl create -f filebeat-kubernetes.yaml

And that's all! I hope you will find this helpful.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.