How to install and configure multiple filebeat in linux instance

Elastic Stack Beats
1

I have already installed one filebeat and it is working fine. I need to create multiple filebeat in the linux instances.

  1. How to start and stop the filebeat if we install multiple filebeat instances?
  2. How to differentiate the registry file?
  3. Want to use the same index for the multiple instances?

Can you please help me with this? This is bit urgent.

2

If you're on a systemd system, you can create a (second) unit file template and you can call as many of them as you like. Each can use a different config file.

systemctl start filebeat@foo.service would use filebeat.foo.yml, or whatever you like.

3

Thanks Dave for your suggestion. I have the below issues..

  1. Unable to install the 2nd filebeat in linux instance stating below
    Preparing packages...
    package filebeat-6.2.4-1.x86_64 is already installed

  2. I tried to install manually using tar and unable to start it.

filebeat.2.yml

systemctl start filebeat@2.service
Failed to start filebeat@2.service: Unit not found.

Please advise.

4

Hi @senthil_murugan there are lots of way to start multiple instances of filebeat in same host if you want to use same version of filebeat.

Start Multiple Instance of Filebeat using "filebeat.sh"

  1. [root@filebeat ~]# filebeat.sh -path.config [mention new config file path] -path.data [mention your new registry file path]

[Note : You can execute the above in different shell with different parameters. You can also check the multiple instances of filebeat using "#ps -ax | grep filebeat" command. But in this case it is dependent on your shell and will be in blocking state]

Start Multiple Instance of Filebeat using "filebeat-god" daemonizer

1.[root@filebeat ~]# cd /usr/share/filebeat/bin
2.[root@filebeat ~]# ./filebeat-god -f -r / -n -p /var/run/filebeat.pid -- /usr/share/filebeat/bin/filebeat -path.config [mention new config filepath] -path.data [mention registry filepath]

[Remarks : Execute step 2 multiple times with different parameter's value, to create multiple instances of filebeat. Verify it using #ps -ax | grep filebeat ]

[Note : “filebeat-god” is a helper, used to daemonize the Filebeat process because Golang processes can't easily daemonize themselves. It is not much required where “systemd” is running. It doesn't supervise the program, just run it as a daemon and takes care of its console output with minimal resources.

The “filebeat” process is handled by “filebeat-god”. If user kill filebeat then “filebeat-god” will be automatically destroyed]

Note: If you want to daemonize multiple instances of filebeat and overcome the problems of starting filebeat from multiple shells then prefer filebeat-god :grinning:

Start Multiple Instances using "Systemctl"
1.[root@filebeat ~]# cd /usr/lib/systemd/system
2. [root@filebeat ~]# cp filebeat.service filebeat1.service
3. [root@filebeat ~]# vim filebeat1.service

Change the paths according to your requirement like below
ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/secondinstance/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat/secondinstance -path.data /var/lib/filebeat2 -path.logs /var/log/filebeat2

4.[root@filebeat ~]# systemctl start filebeat1.service
5.[root@filebeat ~]# systemctl start filebeat.service

I think this will be helpful to you

9 Likes
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.