Installation of module netflow in Logstash 6.5.4

Hi All

I'm trying to install the module netflow for logstash with the online guide https://www.elastic.co/guide/en/logstash/current/netflow-module.html

but I have the problem. I stop the service logstash on my centos linux system and then I changed the logstash.yml adding the module

I attach the configuration file
modules:

  • name: netflow
    var.elasticsearch.hosts: "elasticsearch.neteyelocal:9200"
    var.elasticsearch.username: "logstash"
    var.elasticsearch.password: "my password"
    var.elasticsearch.ssl.certificate: /neteye/shared/logstash/conf/admin.crt.pem
    var.elasticsearch.ssl.key: /neteye/shared/logstash/conf/admin.key.pem
    var.elasticsearch.ssl.certificate_authority: /neteye/shared/logstash/conf/root-ca.crt
    var.elasticsearch.ssl.enabled: true
    var.kibana.scheme: "https"
    var.kibana.host: "kibana.neteyelocal"
    var.kibana.username: "kibanaserver"
    var.kibana.password: "mypassword"
    var.kibana.ssl.certificate: /neteye/shared/kibana/conf/certs/kibana.neteyelocal.crt.pem
    var.kibana.ssl.key: /neteye/shared/kibana/conf/certs/private/kibana.neteyelocal.key.pem
    var.kibana.ssl.certificate_authority: /neteye/shared/kibana/conf/certs/root-ca.crt
    var.input.udp.port: 9996

Then I changed the startup option for logstash changing the file

Then I start the logstash with the command line

/usr/share/logstash/bin/logstash --setup --path.settings=/neteye/shared/logstash/conf

in order to setup the dahsboard, index and other component.

I see the log of Elastic and I don't have error and the log of logstash too. I attach the log of logstash. Part of file logstash-plain.log

[2019-04-04T15:05:48,530][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-04-04T15:05:48,681][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.5.4"}
[2019-04-04T15:05:56,135][INFO ][logstash.config.modulescommon] Setting up the netflow module
[2019-04-04T15:06:02,216][ERROR][logstash.modules.kibanaclient] Error when executing Kibana client request {:error=>#<Manticore::ClientProtocolException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}
[2019-04-04T15:06:05,070][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>LogStash::ConfigLoadingError, :message=>"Failed to parse the module configuration: [elasticsearch.neteyelocal:9200 failed to respond]", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in block in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:incall'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in call_once'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:incode'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:inperform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:inperform_request'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:79:in head'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:49:incan_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:133:in can_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:82:inblock in pipeline_configs'", "org/jruby/RubyArray.java:1734:in each'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:54:inpipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/modules.rb:14:in pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:61:inblock in fetch'", "org/jruby/RubyArray.java:2481:in collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:60:infetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:150:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:101:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:inblock in initialize'"]}
[2019-04-04T15:06:05,200][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>RuntimeError, :message=>"Could not fetch the configuration, message: Failed to parse the module configuration: [elasticsearch.neteyelocal:9200 failed to respond]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:157:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:101:inexecute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:inblock in initialize'"]}
[2019-04-04T15:06:10,740][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

Thank you
Franco

Some suggestion?
Could you show how use command line and pass the certification key?

Thank you
Franco

I pass the problem of certification path. Now I have a different problem. At the end of the installation of the module, in the pipelines parts I think I have this continuous warning (I wait different hour but the setup of the module didn't finish)

[2019-04-06T00:06:51,252][INFO ][logstash.inputs.udp ] UDP lis
tener started {:address=>"0.0.0.0:9996", :receive_buffer_bytes=>"21
2992", :queue_size=>"2000"}
[2019-04-06T00:07:08,041][WARN ][logstash.outputs.elasticsearch] At
tempted to resurrect connection to dead ES instance, but got an err
or. {:url=>"https://logstash:xxxxxx@elasticsearch.neteyelocal:9200/
", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool:
:BadResponseCodeError, :error=>"Got response code '401' contacting
Elasticsearch at URL 'https://elasticsearch.neteyelocal:9200/'"}
[2019-04-06T00:07:14,163][WARN ][logstash.outputs.elasticsearch] At
tempted to resurrect connection to dead ES instance, but got an err
or. {:url=>"https://logstash:xxxxxx@elasticsearch.neteyelocal:9200/
", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool:
:BadResponseCodeError, :error=>"Got response code '401' contacting
Elasticsearch at URL 'https://elasticsearch.neteyelocal:9200/'"}

I am in debug mode. I try to change the user and password and I used the elastic user with admin user but I have the same result.

I check the status of elastic with curl and I have this result

curl -u admin:mypassword "https://elasticsearch.neteyelocal:9200"
{
"name" : "neteye4.test.it",
"cluster_name" : "neteye",
"cluster_uuid" : "fziVBaZcSgK6PjZ_-aecJQ",
"version" : {
"number" : "6.5.4",
"build_flavor" : "oss",
"build_type" : "rpm",
"build_hash" : "d2ef93d",
"build_date" : "2018-12-17T21:17:40.758843Z",
"build_snapshot" : false,
"lucene_version" : "7.5.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

I hope to receive a feedback on that.

Thank you
Franco

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.