# Run "---setup" netflow module when LS is a service (was Install netflow module error) **URL:** \ **Category:** Logstash\ **Created:** [November 27, 2017, 5:17am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166 "2017-11-27T05:17:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 27, 2017, 5:17am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/1 "2017-11-27T05:17:27Z")
ENV: Cluster 6.0, LS 6.0, x-pack trial, Cent0s 7 When i run install module netflow i got error ``` /usr/share/logstash/bin/logstash --path.settings=/etc/logstash --modules netflow --setup -M "netflow.var.kibana.host=http://kubana-url:5601" -M "netflow.var.output.kibana.username=kibana" -M "netflow.var.output.kibana.password=my-password" -M "netflow.var.input.udp.port=9996" -M "netflow.var.elasticsearch.host=http://10.1.1.12:9200" -M "netflow.var.elasticsearch.user=elastic" -M "netflow.var.elasticsearch.password=my-password" 2017-11-27 12:14:36,672 main ERROR Unable to locate appender "${sys:ls.log.format}_rolling" for logger config "root" 2017-11-27 12:14:37,849 main ERROR Unable to locate appender "${sys:ls.log.format}_rolling" for logger config "root" Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties ERROR: Settings 'path.config' (-f) or 'config.string' (-e) can't be used in conjunction with (--modules) or the "modules:" block in the logstash.yml file. usage: bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG] bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE"] [-t] [-w COUNT] [-l LOG] bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG] bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace] bin/logstash -V [--log.level fatal|error|warn|info|debug|trace] bin/logstash --help ``` What happen ? ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 27, 2017, 12:03pm UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/2 "2017-11-27T12:03:36Z")
I take it that you have installed logstash as a service and are trying to run the `setup` step. Yes? ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 27, 2017, 4:10pm UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/3 "2017-11-27T16:10:57Z")
@tatdat ^ question ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 7:23am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/4 "2017-11-28T07:23:31Z")
@guyboertje Yeah, i installed logstash success and it worked with my config. Now i want run setup module netflow (import dashboard, search, visualization for kibana, dictionary, geoip, filter for logstash). ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 7:59am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/5 "2017-11-28T07:59:55Z")
This is the process that we brainstormed. 1. Stop the service `sudo systemctl stop logstash.service` 2. Locate the various settings files for logstash, see [https://www.elastic.co/guide/en/logstash/current/dir-layout.html#deb-layout](https://www.elastic.co/guide/en/logstash/current/dir-layout.html#deb-layout) 3. Edit the `logstash.yml` to specify all the modules settings, so you don't need to put them on the cmd line, see [https://www.elastic.co/guide/en/logstash/current/logstash-modules.html#setting-logstash-module-config](https://www.elastic.co/guide/en/logstash/current/logstash-modules.html#setting-logstash-module-config) 4. Edit the `/etc/logstash/startup.options` file, look for the `LS_OPTS` entry and add `--setup` 5. Start the service 6. check in Kibana that the dashboards are uploaded. 7. Stop the service. 8. Remove the `--setup` option from `LS_OPTS`. 9. Start the service. Report back whether it worked or not. ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 9:47am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/6 "2017-11-28T09:47:00Z")
I followed your guide but when start service , nothing happen. I checked with ps command, i saw logstash is running but when i check log file (logstash-plain.log), nothing and of course, nothing happen in kibana. This is [my config](https://pastebin.com/zXnhRC0Y) P/s: I setup kibana running in port 443 with https ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 9:50am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/7 "2017-11-28T09:50:19Z")
Please post your `logstash.yml` file contents here (inside two triple backtick ``` lines) ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 9:50am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/8 "2017-11-28T09:50:52Z")
> [@guyboertje](#): > > Please post your logstash.yml file contents here (inside two triple backtick ``` lines) Yeah i updated [my config](https://pastebin.com/zXnhRC0Y) ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 10:07am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/9 "2017-11-28T10:07:14Z")
Make sure that: `var.kibana.scheme` is set to "https" `var.kibana.host` **does not** contain a "https://" prefix. `var.elasticsearch.hosts` **does** contain a "https://" prefix ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 10:14am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/10 "2017-11-28T10:14:11Z")
Yeah, i added `var.kibana.scheme` to my config but still got same result. I'm wondering why I can not see the log file. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 10:19am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/11 "2017-11-28T10:19:28Z")
In logstash.yml set the log.level to debug Then do steps 4-7 and post the full log contents here. ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 10:30am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/12 "2017-11-28T10:30:35Z")
Here is my [log file](https://pastebin.com/gf38U7eG) ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 11:17am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/13 "2017-11-28T11:17:03Z")
This entry tells me that the `LS_OPTS` method is not working. ```auto [2017-11-28T17:28:17,187][DEBUG][logstash.runner] modules_setup: false ``` I did not tell you to do (from the `startup.options file`): ```auto ################################################################################ # These settings are ONLY used by $LS_HOME/bin/system-install to create a custom # startup script for Logstash and is not used by Logstash itself. It should # automagically use the init system (systemd, upstart, sysv, etc.) that your # Linux distribution uses. # # After changing anything here, you need to re-run $LS_HOME/bin/system-install # as root to push the changes to the init script. ################################################################################ ``` revised steps: 1. Stop the service sudo systemctl stop logstash.service 2. Locate the various settings files for logstash, see [https://www.elastic.co/guide/en/logstash/current/dir-layout.html#deb-layout](https://www.elastic.co/guide/en/logstash/current/dir-layout.html#deb-layout) 3. Edit the logstash.yml to specify all the modules settings, so you don't need to put them on the cmd line, see [https://www.elastic.co/guide/en/logstash/current/logstash-modules.html#setting-logstash-module-config](https://www.elastic.co/guide/en/logstash/current/logstash-modules.html#setting-logstash-module-config) 4. Edit the /etc/logstash/startup.options file, look for the LS\_OPTS entry and add --setup 5. Run $LS\_HOME/bin/system-install as root 6. Start the service 7. check in Kibana that the dashboards are uploaded. 8. Stop the service. 9. Remove the --setup option from LS\_OPTS. 10. Run $LS\_HOME/bin/system-install as root 11. Start the service. ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 11:38am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/14 "2017-11-28T11:38:58Z")
Thank @guyboertje. I followed your guilde and re check kibana. Still got nothing Here is log file with debug level [https://pastebin.com/gf38U7eG](https://pastebin.com/gf38U7eG) ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [November 28, 2017, 12:45pm UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/15 "2017-11-28T12:45:28Z")
So now the setup is set to true. Also, In the logs I see: "var.elasticsearch.hosts"=\>"10.1.6.195:9200" -\> no scheme "var.kibana.host"=\>"[log.ho.fpt.vn](http://log.ho.fpt.vn)" -\> no port "var.kibana.scheme"=\>"https" But in your original post you had: /usr/share/logstash/bin/logstash --path.settings=/etc/logstash --modules netflow --setup -M "netflow.var.kibana.host=http://kubana-url:5601" -M "netflow.var.output.kibana.username=kibana" -M "netflow.var.output.kibana.password=my-password" -M "netflow.var.input.udp.port=9996" -M "netflow.var.elasticsearch.host=http://10.1.1.12:9200" -M "netflow.var.elasticsearch.user=elastic" -M "netflow.var.elasticsearch.password=my-password" What port is your Kibana server running behind? What is the scheme for Elasticsearch and Kibana? Is it `http` or `https`? Is `log.ho.fpt.vn` the Kibana server or a proxy of some kind? ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [November 28, 2017, 2:31pm UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/16 "2017-11-28T14:31:46Z")
> [@guyboertje](#): > > What port is your Kibana server running behind? > > What is the scheme for Elasticsearch and Kibana? Is it http or https? > > Is log.ho.fpt.vn the Kibana server or a proxy of some kind? Kibana server running in port 443 witl ssl enabled (dont behind reverse proxy) schema for ES is http, kibana is https I'm trying to setting kibaan running in default port 5601 and scheme is http but still can't setup netflow module. p/s: i'm testing with x-pack, enable TLS transport between nodes. Cluster ES dont have any data. ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [December 12, 2017, 12:58pm UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/17 "2017-12-12T12:58:11Z")
i still can't setup netflow module 😢 Can someone help me? ---
**Author:** ![antony.y](https://avatars.discourse-cdn.com/v4/letter/a/91b2a8/32.png) [@antony.y](https://discuss.elastic.co/u/antony.y)\ **Post date:** [December 13, 2017, 6:29am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/18 "2017-12-13T06:29:54Z")
i have tried that , it seems that now in my version (5.x),i can using 'systemctl start logstash' to start the logstash, and it does work, i can see all the port i've configured in my conf.d , but only netflow module can take effect.So i tried to use command line to achieve the function i want. ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/4/846242a0cbe49294fe6fa2b4ab97a5ba10417487.png) ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [December 13, 2017, 6:57am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/19 "2017-12-13T06:57:01Z")
@antony.y, I try to run with command but not success `/usr/share/logstash/bin/logstash --modules netflow --setup -M "netflow.var.kibana.host=http://10.1.11.115:5601" -M "netflow.var.output.kibana.username=username" -M "netflow.var.output.kibana.password=password" -M "netflow.var.input.udp.port=9996" -M "netflow.var.elasticsearch.hosts=http://10.1.11.115:9200" -M "netflow.var.elasticsearch.user=username" -M "netflow.var.elasticsearch.password=password"` I got some error > WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults > Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console > [ERROR] 2017-12-13 13:53:10.439 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] sourceloader - Could not fetch all the sources {:exception=\>LogStash::ConfigLoadingError, :message=\>"Failed to parse the module configuration: [[401] ]", :backtrace=\>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:202:in `__raise_transport_error'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:319:in `perform\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/client.rb:131:in `perform\_request'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch\_client.rb:85:in `head'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch_client.rb:55:in `can\_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/elasticsearch\_client.rb:139:in `can_connect?'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:76:in `block in pipeline\_configs'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/config/modules_common.rb:56:in `pipeline\_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source/modules.rb:16:in `pipeline_configs'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:59:in `block in fetch'", "org/jruby/RubyArray.java:2481:in `collect'", "/usr/share/logstash/logstash-core/lib/logstash/config/source_loader.rb:58:in `fetch'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:148:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]} > [ERROR] 2017-12-13 13:53:10.444 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - An exception happened when converging configuration {:exception=\>RuntimeError, :message=\>"Could not fetch the configuration, message: Failed to parse the module configuration: [[401] ]", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/agent.rb:155:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]} ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [December 13, 2017, 7:02am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166/20 "2017-12-13T07:02:43Z")
Yeah i found m problem, Some parameter in command not right in Logstash 6.x It worked with `/usr/share/logstash/bin/logstash --modules netflow --setup -M "netflow.var.kibana.host=10.1.11.115:5601" -M "netflow.var.kibana.username=username" -M "netflow.var.kibana.password=password" -M "netflow.var.input.udp.port=9996" -M "netflow.var.elasticsearch.hosts=10.1.11.115:9200" -M "netflow.var.elasticsearch.username=username" -M "netflow.var.elasticsearch.password=password"` [Next page](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166.md?page=2)