Hi thanks for the wonderful elastic stack! I am trying to monitor the SLOWLOG of Redis in elastic, and I am on Kubernetes (using ECK). The doc says Redis filebeat for slowlog needs to specify a list of hosts. However, in k8s what hosts should I write down?
Say I have 10 nodes and 30 redis master-replica pairs. If I simply write down all 30 redis uris, then EACH of the 10 nodes will try to collect data. That is a disaster - 10x redundancy!
How are you configuring Filebeat? With autodiscover you can make Filebeat to discover automatically your running redis containers and add an only configuration for them, so you don't need to manually add additional entries for each one of your hosts. As you said, having to configure the list of hosts on all filebeat would have bad consequences, apart of being quite cumbersome
With autodiscover It would be something like this (not tested, but you can get the idea):
This will start a redis module for each container with a image that contains "redis" on its name. The configuration includes some variables that will be replaced, in this case with the container id, and with its IP respectively.
Thanks very much! Now I see how to implement it However I do not know how to safely add password to it? For example, I can put password in annotations and let elastic read it. But annotations are very unsafe to put password. Can I read k8s Secret from elastic? Thanks!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.