Multi-cluster Kubernetes monitoring - best practices? Cluster name field?

Elastic Stack Beats
1

Hi,

I want to monitor multiple Kubernetes Clusters to one ES cluster using Filebeats.

Are there any best practices on how to separate the monitored clusters?

Can I add custom field to Beats per cluster? So it'll send something like "cluster_name" so I can filter by it?

Thanks!

2

Excatly. You can add a custom field to your filebeat.yml of a cluster.

fields:
  cluster_name: my_cluster

Or to collect the logs of a cluster in the same index in ES, you can set a unique index name:

index: "filebeat-%{[beat-version]}-my-cluster-id-%{[+yyyy.MM.dd]}"
setup.template.name: "filebeat-%{[beat-version]}-my-cluster-id"
setup.template.pattern: "filebeat-%{[beat-version]}-my-cluster-id-*"
3

Followup question, now that I have all the cluster logs in the elasticsearch, separated with cluster_name tag - how can I take specific logs coming from 3 specific kubernetes apps and parse them into separate indexes?

Currently I have both "system" logs for the VPN, Calico and all other logs combined with the application logs.

Thanks