Logstash with multiple conf files

[Kfiro (Kfir)]

Hi ELK,
I'm trying to configure my Logstash to run with 2 "conf" files,

First: winlogbeat.conf

input {
  beats {
    port => 5044
  }
}
output {
		elasticsearch {
				hosts=>["https://elk01:9200"]
				index=>"logstash-%{+YYYY.MM.dd}b"
				cacert => '/etc/logstash/config/certs/ca.crt'
				user => 'elastic'
				password => '123456'
		}
}

Second: packetbeat.conf

input {
  beats {
    port => 5045
  }
}
output {
		elasticsearch {
				hosts=>["https://elk02:9200"]
				index=>"packetbeat-%{+YYYY.MM.dd}b"
				cacert => '/etc/logstash/config/certs/ca.crt'
				user => 'elastic'
				password => '123456'
		}
}

This is my logstash.yml:

path.data: /var/lib/logstash
pipeline.ordered: auto
path.logs: /var/log/logstash
log.level: debug
node.name: logstashp01
path.config: /etc/logstash/conf.d/*.conf

and this the pipline.yml:

• pipeline.id: pipeline_1
  path.config: "/etc/logstash/conf.d/winlogbeat.conf"

• pipeline.id: pipeline_2
  path.config: "/etc/logstash/conf.d/packetbeat.conf"

Then i'm starting Logstash from command Line:
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/examplelk01.conf,/etc/logstash/conf.d/packetbeat.conf

Logstash's failed to run

can you help me please... what am i missing or any otherway to solve that issue ??

[warkolm (Mark Walkom)]

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you

You haven't described what problem you are trying to solve, so it's not clear what the issue is here sorry.

[Badger]

You should be getting the warning

[WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

Remove -f /etc/logstash/conf.d/examplelk01.conf,/etc/logstash/conf.d/packetbeat.conf from the command line. logstash will then read pipelines.yml and run each configuration file in its own pipeline.

If you keep the -f then logstash will concatenate the configuration files into a single configuration. It will read events from both beats inputs, and send them all to both elasticsearch outputs.

[Kfiro (Kfir)]

Thanks, I Edited the post.

[Kfiro (Kfir)]

Thank you Badger,
Its work!!

only needed to add --path.settings /etc/logstash on the command line

[Kfiro (Kfir)]

@Badger
thank you for the response friend,
the logstash is running with the 2 config files,
but both the winlogeat agent and the packetbeat agent are writing events to the 2 indices:
logstash and packetbeat,
Although I have set in both Conf Files to send to a separate index as you can see

do you know why?
Thanks in advance

[Badger]

As I said, if you use -f then logstash will concatenate the configuration files into a single configuration. It will read events from both beats inputs, and send them all to both elasticsearch outputs. Similarly if you set path.config in pipelines.yml to include both configurations for one pipeline.