How to enable symlinks?

Elastic Stack Beats
#1

Hi there!

I'm pretty much interested in enabling the symlinks option for the reason already stated at the documentation https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#_literal_symlinks_literal "The symlinks option can be useful if symlinks to the log files have additional metadata in the file name, and you want to process the metadata in Logstash. This is, for example, the case for Kubernetes log files."

I'm trying to collect logs from kubernetes, and I can do it without issues when I'm accessing non symlinks files, that is actually what I want to do, since the symlinks files names are giving me a bunch of extra information (app or system that generated the logs).

I tried to add a line "symlinks: true" in the snippet section of the corresponding collector but this just add the line at the end of filebeat.yml.

How should I write the snippet? Or is there any other way to make this working?

Thanks!

#2

To use symlink: true you have to add it to your prospector configuration, and check that your paths are only configured with the paths to your symlink files.

#3

I'm using a collector sidecar that generates the filebeat.yml from the graylog server. This is why I was trying to include that option using a snippet. Is there any way to write the snippet in order to place that line in the https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html as you mention?

Thx!

#4

Another way to solve this issue would be to include support for the symlinks as for other features https://github.com/Graylog2/graylog-plugin-collector/issues/78

Any help really appreciate!

#5

Yes, it seems more an issue with Greylog then, it should allow to add configuration in the prospector itself, does it have any way to do it?

#6

Nop, AFAIK...

#7

It seems that it there is a way to do it, but putting all prospectors configuration in snippets, see https://community.graylog.org/t/how-to-configure-filebeat-from-graylog-to-parse-json/3226/8

#8

Thx. I just copied the full prospectors configuration in snippets and it worked, but I get the configuration twice since the input/output could not be empty. I will do a pull request with a fix to correct it in a more elegant way.

#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.