Hi community,
at my company we use managed Elastic Agents to collect logs from kubernetes. To figure out which cluster the logs belong, every cluster is labeled with a specific kubernetes label that identifies it.
This week we got a new request to log calico network denies. These denies are logged directly into a log file /var/log/messages on the host OS. We have therefore added the custom log integration and added the following processors in order to only collect calico-packet log entries:
- drop_event:
when:
not:
regexp:
message: ".*calico-packet.*"
- dissect:
tokenizer: "%{} %{} %{}: %{message}"
field: "message"
target_prefix: "dissect"
So far everything works as expect.
However, we now face the challenge of making it easily accessible to our colleagues which log was produced by which cluster. Unfortunately, apart from the host names (aks-somepool-12348843-00001, ...), there is no information on this in the content of the logs extracted from /var/log/messages. Is it somehow possible to pass the tags set in the agent definition ELASTIC_AGENT_TAGS to the logs? This way we could, for example, also skip setting the label on the Kubernetes nodes. If not, I am also open for other suggestions.
Thanks in advance