I have a kubernetes cluster (currently in dev status) where following is running:
- elastic stack
- different project specific apps.
I want to use the same elastic stack for monitoring my kubernetes instance and my project apps. (Yes, I am aware of the fact, if my kubernetes goes down kompletely, my elastic stack will also fall down).
I do not want to ship logs of the elastic stack to elasticsearch, because this insert may lead to more logs in elasticsearch, so I could end up in some infinity loop.
In your sample for running filebeat in kubernetes at https://www.elastic.co/de/blog/shipping-kubernetes-logs-to-elasticsearch-with-filebeat I saw, that the file path of the prospector is simply set to
This would grap all container logs!
What is the most easy and resource efficient way to define dynamically which logs should be shipped to logstash and which not?
I could send everything to logstash and drop it there by checking the metadata. For example I could introduce a label saveLogsInEs=true. If this is set, process the log, otherwise drop it. But I assume this is quite a resource overkill.
Or is there a way to drop in filebeat to drop by kubernetes label before sending to logstash? In this case the log lines are read, enriched, then dropped. I would like to avoid it if possible.
Or at best, is there a possibility to exclude prospector paths by kubernetes labels? I think this would be the optimal way to reduce unneeded load, because the file does not even need to be tailed.
Thanks a lot,