Kibana will not start with netflow module


(waritha) #1

[root@localhost ~]# /usr/share/logstash/bin/logstash --modules netflow --setup -M netflow.var.input.udp.port=9996 -M netflow.var.kibana.host="localhost:5601" -M netflow.var.elasticsearch.hosts="localhost:9200"
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2018-09-05 17:20:15.471 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-09-05 17:20:16.750 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.4.0"}
[INFO ] 2018-09-05 17:20:16.989 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] modulescommon - Setting up the netflow module
[INFO ] 2018-09-05 17:20:50.118 [Converge PipelineAction::Create] pipeline - Starting pipeline {:pipeline_id=>"module-netflow", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-09-05 17:20:50.603 [[module-netflow]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:disappointed_relieved::9200/]}}
[INFO ] 2018-09-05 17:20:50.627 [[module-netflow]-pipeline-manager] elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[WARN ] 2018-09-05 17:20:50.863 [[module-netflow]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"http://localhost:9200/"}
[INFO ] 2018-09-05 17:20:50.888 [[module-netflow]-pipeline-manager] elasticsearch - ES Output version determined {:es_version=>6}
[WARN ] 2018-09-05 17:20:50.915 [[module-netflow]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>6}
[INFO ] 2018-09-05 17:20:50.987 [[module-netflow]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[INFO ] 2018-09-05 17:20:52.558 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}
[INFO ] 2018-09-05 17:20:52.726 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"}
[INFO ] 2018-09-05 17:20:52.811 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}
[INFO ] 2018-09-05 17:20:52.812 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"}
[INFO ] 2018-09-05 17:20:52.813 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}
[INFO ] 2018-09-05 17:20:52.813 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"}
[INFO ] 2018-09-05 17:20:52.815 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}
[INFO ] 2018-09-05 17:20:52.815 [[module-netflow]-pipeline-manager] geoip - Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"}
[INFO ] 2018-09-05 17:20:53.027 [Converge PipelineAction::Create] pipeline - Pipeline started successfully {:pipeline_id=>"module-netflow", :thread=>"#<Thread:0x5f21e97b run>"}
[INFO ] 2018-09-05 17:20:53.107 [[module-netflow]<udp] udp - Starting UDP listener {:address=>"0.0.0.0:9996"}
[INFO ] 2018-09-05 17:20:53.303 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:"module-netflow"], :non_running_pipelines=>[]}
[INFO ] 2018-09-05 17:20:53.604 [[module-netflow]<udp] udp - UDP listener started {:address=>"0.0.0.0:9996", :receive_buffer_bytes=>"212992", :queue_size=>"2000"}
[INFO ] 2018-09-05 17:20:54.022 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[WARN ] 2018-09-05 17:21:01.424 [<udp.0] netflow - Can't (yet) decode flowset id 256 from source id 0, because no template to decode it with has been received. This message will usually go away after 1 minute.


(Nathan Reese) #2

What version of kibana and logstash are you using?

The error message provided looks like the output of the logstash process. Can you provide the output of the Kibana process?


(waritha) #3

kibana and logstash version are 6.4.0-1


(Nathan Reese) #4

So it looks like Kibana has successfully started based on the screen shot. What is the error you are seeing?


(waritha) #6

i have already config logstash netflow module but not show on kibana


(Robert Cowart) #7

You may want to consider using ElastiFlow. The Logstash Netflow Module was based on ElastiFlow 1.0.0 and is very behind now. You will be able to handle a larger variety of flows sources and get more value from the data by using ElastiFlow.


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.