I've attempted to implement kubernetes logging to elasticsearch.
What I want to achieve:
Ship all the logs from pods in the default namespace.
Configure templates per pod type
That was my first attempt and it works (ships all pods ). Then I've decided to enable autodiscover option and limit it to only log pods from default namespace. Here is my config:
Unfortunately it doesn't log anything at all. Even though I have a pod in the default namespace there is nothing shipped to ES. If I uncomment hints.enabled it logs everything (both logs from my desired pod and filebeat logs - filebeat is form the kube-system namespace)
Is it enough information to determinate what might be a problem?
What I've been able to find out is that logs in my cluster are located in a different folder
For example: /var/lib/docker/containers/64b4e054eb0007977ec7124a326dbbd3c257c9e75ceb3b8c0d611e7851f10a4d/64b4e054eb0007977ec7124a326dbbd3c257c9e75ceb3b8c0d611e7851f10a4d-json.log
I've just a newly created AWS EKS cluster (Kubernetes version 1.14), newly started ES and Kibana (version 7.4), running outside of the cluster.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.