Multi-cluster Kubernetes monitoring - best practices? Cluster name field?


I want to monitor multiple Kubernetes Clusters to one ES cluster using Filebeats.

Are there any best practices on how to separate the monitored clusters?

Can I add custom field to Beats per cluster? So it'll send something like "cluster_name" so I can filter by it?


Excatly. You can add a custom field to your filebeat.yml of a cluster.

  cluster_name: my_cluster

Or to collect the logs of a cluster in the same index in ES, you can set a unique index name:

index: "filebeat-%{[beat-version]}-my-cluster-id-%{[+yyyy.MM.dd]}" "filebeat-%{[beat-version]}-my-cluster-id"
setup.template.pattern: "filebeat-%{[beat-version]}-my-cluster-id-*"

Followup question, now that I have all the cluster logs in the elasticsearch, separated with cluster_name tag - how can I take specific logs coming from 3 specific kubernetes apps and parse them into separate indexes?

Currently I have both "system" logs for the VPN, Calico and all other logs combined with the application logs.


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.