Logstash configuration problem

[thierry-decker]

I'm currently working on a Logstash configuration.

Working on version 6.1.

My problem is that I want my logstash instance to receive events from both NetFlow module and rsyslog senders.

Configuring NetFlow module in logstash.yml perfectly works alone.
Configuring a syslog collector via pipelines.yml and a configuration file in conf.d perfectly works alone.

As soon as I want to load Netflow module via logstash.yml, the pipielines.yml configuration in conf.d directory is simply ignored.

Is there a way to have both services running on a unique Logstash instance and if so, what can be the good practice ?

[guyboertje]

Modules were designed as a quick start solution to allow users to move past the need to learn the LS config language and onto experimenting with ES and Kibana objects sooner because the dashboards are pre-built.

When you get to the stage of wanting to run multiple pipelines you have moved past the quick start situation.

the Netflow module is opensource at https://github.com/elastic/logstash/tree/master/modules/netflow/configuration/logstash
The file netflow.conf.erb uses template substitution. Look for the parts <%= some setting %> and replace it with real settings and remove the .erb part of the file name.
Add an elastic search output to the config. use log.level=debug with -t --modules=netflow and you will see the what various settings for the ES output is as DEBUG log messages.

When you have done that you can then run the netflow config but not as a module.

[thierry-decker]

Thanks for your quick answer!

I spent nearly a full day on this and you saved my time.