Filebeat log to multiple outputs like file and syslog

Elastic Stack Beats
1

Hi,

I'm using filebeat on Linux in this version:

$ rpm -qa | grep filebeat
filebeat-8.7.0-1.x86_64

I would like to log filebeat to logfiles and also to syslog. This is the configuration snippet:

logging:
  to_files: true
  to_syslog: true
  files:
    name: filebeat
    rotateeverybytes: 10485760
    keepfiles: 2
  metrics:
    enabled: false

path:
  logs: /var/lib/graylog-sidecar/collectors/filebeat/log
...

Logging to syslog works fine ...

Jun 12 09:42:14 XXX filebeat[8839]: 2023-06-12T09:42:14.423+0200 INFO [publisher_pipeline_output] map[file.line:139 file.name:pipeline/client_worker.go] Connecting to backoff(async(tcp://loginput-t02.lvm.de:5046))#011{"ecs.version": "1.6.0"}

... but there are no logfiles:

$ ls -alh /var/lib/graylog-sidecar/collectors/filebeat/log
total 0
drwx------ 2 graylog graylog  6 Jun 12 09:41 .
drwxr-xr-x 4 graylog graylog 29 Jun  6 16:48 ..

Two questions:

  1. Can only one log destination be used or should my configuration work?
  2. Is it possible to change filebeats log format for syslog logging to json?
2

Hi @michaelbu,

Multiple outputs for a single running instance of filebeat isn't supported. There is a discussion from 2021 here. There is a workaround posted here where you can run 2 filebeats each pointing to a different output. Another alternative would be to use Logstash which does support multiple outputs.

For your second question, what do you mean by:

Have you had a look at the syslog processor to see if that does want you want to do?

3

Hi @carly.richmond,

thanks for your answer.

I think this is a misunderstanding: I do not want to send the logs to different output. I would like to configure the filebeat to log it's own messages both to files (the deafult) and to syslog.

Do you think this filebeat configuration snippet is valid?

...
logging:
  to_files: true
  to_syslog: true
...

I hope this clarifies my problem?

Regards
Michael

4

Hi,

is filebeat designed to send it's own logging only to one technology to_files or to_syslog?

Or is this a valid configuration?

logging.to_files: true
logging.to_syslog: true
5

Hi @michaelbu,

I've been digging through the beats repo as the docs suggest it's one or the other to me. I'm not sure logging to both is valid, but I'm double checking to see if I can get an answer for you.

6

Hi @carly.richmond,

but I'm double checking to see if I can get an answer for you.

Thank you, that will be great!

7

Hi @michaelbu,

Got an answer for you. For beats you can only specify one log output destination, not two. The relevant code is here:

Hope that helps!

8

Hi @carly.richmond,

thank you for this clarifying this!

I think it'll confuses me a little bit. If only one destination is allowed, I'd expected a configuration like this:

logging.to: files

or alternativly

logging.to: syslog

Thank you very much!

Regards
Michael