Is it possible to collect the file owner (username) for a log file?

Elastic Stack Beats
1

We are using filebeat to collect hadoop / yarn userlogs when running yarn applications.

Currently, our hadoop/yarn setup uses hadoop's authentication mechanisms, and therefore, the userlogs (normally under /var/log/yarn/userlogs/**) we collect from hadoop are owned by individual authenticated users.

Is it possible or would be possible to allow filebeat to collect the username (and perhaps optionally, the group name) that of the user/group of the file owner?

If/when this would be collected, we could then easily filter for individual users when exploring our logs via elasticsearch / kibana....

2

This is currently not possible. I remember seeing similar requests in the past but couldn't find it Github, only this one here: https://github.com/elastic/beats/issues/4169 The reason I think it's similar as it also requests to add metadata about the file to each event. We could potentially solve this with a add_file_metata processor.

Do you want to open a Github issue with this idea?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.