Logstash with a netflow module AND other config files

Hello,

I'm wandering if it's possible to use logstash with an imported module along side with other configuration files.

I'm using the netflow module. After a first setup (described here : https://www.elastic.co/guide/en/logstash/current/netflow-module.html).

I added in my logstash.yml the following lines :

//////////// logstash.yml /////////

modules:

  • name: netflow
    //////////////////////////////

Every thing worked finely.

Then I tried to add my own config files :

  1. If I use the "path.config" inside the logstash.yml file ( path.config: /etc/logstash/conf.d/*.conf) , I get an error in my logs :

[2018-07-26T12:16:49,669][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

  1. If I remove the "path.config" directive from logstash.yml and use the pipeline.yml file to do so :
  • pipeline.id: main
    path.config: "/etc/logstash/conf.d/*.conf"

The file is ignored :
[2018-07-26T12:52:01,041][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-07-26T12:52:01,589][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.0"}
[2018-07-26T12:52:01,697][INFO ][logstash.config.modulescommon] Starting the netflow module
[2018-07-26T12:52:14,264][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"module-netflow", "pipeline.workers"=>48, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5
0}

How to do to have Logstash running module sources along with other sources (with their own config files) ?

Thank you in advance.

I'm wandering if it's possible to use logstash with an imported module along side with other configuration files.

I don't think that's allowed.

Thank you.
I finally went with 2 instances of logstash and it works.

@samia, you may want to consider ElastiFlow (https://github.com/robcowart/elastiflow) for your netflow collection. The Logstash Netflow Module was based on v1.0.0 of ElastiFlow and is now quite old. ElastiFlow can also be configured using pipelines.yml and run together with other pipelines in a single logstash instance.

1 Like

@rcowart Thank you for your advice !
I'll test your solution.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.