@guyboertje
Yeah, i installed logstash success and it worked with my config. Now i want run setup module netflow (import dashboard, search, visualization for kibana, dictionary, geoip, filter for logstash).
I followed your guide but when start service , nothing happen. I checked with ps command, i saw logstash is running but when i check log file (logstash-plain.log), nothing and of course, nothing happen in kibana.
This is my config
P/s: I setup kibana running in port 443 with https
I did not tell you to do (from the startup.options file):
################################################################################
# These settings are ONLY used by $LS_HOME/bin/system-install to create a custom
# startup script for Logstash and is not used by Logstash itself. It should
# automagically use the init system (systemd, upstart, sysv, etc.) that your
# Linux distribution uses.
#
# After changing anything here, you need to re-run $LS_HOME/bin/system-install
# as root to push the changes to the init script.
################################################################################
revised steps:
Stop the service sudo systemctl stop logstash.service
So now the setup is set to true.
Also, In the logs I see:
"var.elasticsearch.hosts"=>"10.1.6.195:9200" -> no scheme
"var.kibana.host"=>"log.ho.fpt.vn" -> no port
"var.kibana.scheme"=>"https"
What port is your Kibana server running behind?
What is the scheme for Elasticsearch and Kibana? Is it http or https?
Is log.ho.fpt.vn the Kibana server or a proxy of some kind?
Kibana server running in port 443 witl ssl enabled (dont behind reverse proxy)
schema for ES is http, kibana is https
I'm trying to setting kibaan running in default port 5601 and scheme is http but still can't setup netflow module.
p/s: i'm testing with x-pack, enable TLS transport between nodes.
Cluster ES dont have any data.
i have tried that , it seems that now in my version (5.x),i can using 'systemctl start logstash' to start the logstash, and it does work, i can see all the port i've configured in my conf.d , but only netflow module can take effect.So i tried to use command line to achieve the function i want.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[ERROR] 2017-12-13 13:53:10.439 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] sourceloader - Could not fetch all the sources {:exception=>LogStash::ConfigLoadingError, :message=>"Failed to parse the module configuration: [[401] ]", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:202:in __raise_transport_error'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:319:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/http/manticore.rb:67:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/client.rb:131:in perform_request'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:85:in head'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:55:in can_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:139:in can_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:76:in block in pipeline_configs'", "org/jruby/RubyArray.java:1734:in each'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:56:in pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/modules.rb:16:in pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:59:in block in fetch'", "org/jruby/RubyArray.java:2481:in collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:58:in fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}
[ERROR] 2017-12-13 13:53:10.444 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - An exception happened when converging configuration {:exception=>RuntimeError, :message=>"Could not fetch the configuration, message: Failed to parse the module configuration: [[401] ]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.