# Logstash defaults modules? **URL:** \ **Category:** Logstash\ **Created:** [December 19, 2017, 4:40pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476 "2017-12-19T16:40:04Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 19, 2017, 4:40pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/1 "2017-12-19T16:40:04Z")
I just installed ELK and I got it to work for a few minutes then I booted the vm and now Im trying to get back. logstash doesnt listen to the port I specified in the conf file. I also noticed int the log file :/etc/logstash$ tail -f /var/log/logstash/logstash-plain.log [2017-12-19T17:34:03,898][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"} [2017-12-19T17:34:18,708][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [2017-12-19T17:34:18,714][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [2017-12-19T17:34:18,848][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"} [2017-12-19T17:34:33,682][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [2017-12-19T17:34:33,688][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [2017-12-19T17:34:33,849][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"} [2017-12-19T17:34:48,473][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [2017-12-19T17:34:48,479][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [2017-12-19T17:34:48,650][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"} [2017-12-19T17:35:03,288][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [2017-12-19T17:35:03,294][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [2017-12-19T17:35:03,430][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"} two modules are spamming netflow and fb\_apache ? I didn't install netflow nor fb\_apache. this spamming of the logs keeps me from viewing more important log messages. How can I disable these modules? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 19, 2017, 4:47pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/2 "2017-12-19T16:47:26Z")
It looks like you may have put jvm.options into the configuration directory? ---
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 19, 2017, 4:49pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/3 "2017-12-19T16:49:53Z")
how can I verify and undo that? btw logstash is taking up lots of cpu and I havent started to push syslog into it yet, only from one Cisco ASA context. Is this normal? top - 17:50:25 up 3:03, 3 users, load average: 4,16, 4,46, 4,48 Tasks: 242 total, 1 running, 241 sleeping, 0 stopped, 0 zombie %Cpu(s): 0,1 us, 1,0 sy, 20,3 ni, 78,7 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 44723 logstash 39 19 5121708 290676 17580 S 614,6 1,8 0:28.93 java 16vCPU. Edit: I want to clarify that I havent moved any files. until now I moved jvm.options out of /etc/logstash . ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 19, 2017, 6:33pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/4 "2017-12-19T18:33:15Z")
> btw logstash is taking up lots of cpu and I havent started to push syslog into it yet, only from one Cisco ASA context. Is this normal? Is this after you've gotten jvm.options out of the way so that Logstash actually starts up properly? > Edit: I want to clarify that I havent moved any files. until now I moved jvm.options out of /etc/logstash . So the previous error is gone, yes? ---
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 20, 2017, 11:05am UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/5 "2017-12-20T11:05:56Z")
Actually there were high cpu util right after I finished installing ELK when I was using 4 vCPUs and rigt now when I have less then 1 incomming logg per second ( using tcpdump udp port 5544 on the server that is running ELK ) cpu can high. Right know logstash isn't listening to port 5544 any more ---
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 20, 2017, 1:02pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/6 "2017-12-20T13:02:15Z")
Except spam from netflow and fb\_apache modules in the log cat /var/log/logstash/logstash-plain-2017-12-19.log | grep -vE 'netflow|fb\_apache' results in: [2017-12-19T23:57:11,443][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:57:26,814][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:57:42,242][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:57:58,038][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:58:14,597][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:58:30,642][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:58:45,182][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:58:59,579][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:59:14,623][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:59:29,766][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} [2017-12-19T23:59:44,966][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "} logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled) Active: active (running) since ons 2017-12-20 13:59:21 CET; 3s ago Main PID: 62532 (java) Tasks: 34 Memory: 208.9M CPU: 13.278s CGroup: /system.slice/logstash.service └─62532 /usr/bin/java -Xmx500m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.hom cat /etc/systemd/system/logstash.service [Unit] Description=logstash [Service] Type=simple User=logstash Group=logstash Load env vars from /etc/default/ and /etc/sysconfig/ if they exist. Prefixing the path with '-' makes it try to load, but if the file doesn't exist, it continues onward. EnvironmentFile=-/etc/default/logstash EnvironmentFile=-/etc/sysconfig/logstash ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" Restart=always WorkingDirectory=/ Nice=19 LimitNOFILE=16384 [Install] WantedBy=multi-user.target :/usr/share/elasticsearch$ cat /etc/logstash/logstash.yml | grep -vE ^# path.data: /var/lib/logstash path.config: /etc/logstash path.logs: /var/log/logstash ---
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 20, 2017, 1:27pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/7 "2017-12-20T13:27:06Z")
It seems that logstash cant find logstash.yml , but it is located in the default directory. :/usr/share/logstash$ sudo bin/logstash --log.level=debug -t -f /etc/logstash/conf.d/logstash.conf WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [DEBUG] 2017-12-20 14:24:48.792 [LogStash::Runner] DateFilter - Date filter with format=MMM dd HH:mm:ss, locale=null, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.807 [LogStash::Runner] DateFilter - Date filter with format=MMM dd HH:mm:ss, locale=en-US, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.808 [LogStash::Runner] DateFilter - Date filter with format=MMM d HH:mm:ss, locale=null, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.808 [LogStash::Runner] DateFilter - Date filter with format=MMM d HH:mm:ss, locale=en-US, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.808 [LogStash::Runner] DateFilter - Date filter with format=MMM dd yyyy HH:mm:ss, locale=null, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.809 [LogStash::Runner] DateFilter - Date filter with format=MMM dd yyyy HH:mm:ss, locale=en-US, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.809 [LogStash::Runner] DateFilter - Date filter with format=MMM d yyyy HH:mm:ss, locale=null, timezone=null built as org.logstash.filters.parser.JodaParser [DEBUG] 2017-12-20 14:24:48.809 [LogStash::Runner] DateFilter - Date filter with format=MMM d yyyy HH:mm:ss, locale=en-US, timezone=null built as org.logstash.filters.parser.JodaParser Configuration OK :/usr/share/logstash$ ls /etc/logstash/ conf.d log4j2.properties logstash.yml startup.options 1:/usr/share/logstash$ also I can't use the configtest :/usr/share/logstash$ sudo service logstash configtest logstash: unrecognized service :/usr/share/logstash$ sudo service logstash status ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since ons 2017-12-20 14:16:34 CET; 16min ago Process: 1444 ExecStart=/usr/share/logstash/bin/logstash --path.settings /etc/logstash (code=exited, status=143) Main PID: 1444 (code=exited, status=143) dec 20 14:16:25 systemd[1]: Started logstash. dec 20 14:16:33 systemd[1]: Stopping logstash... dec 20 14:16:34 systemd[1]: logstash.service: Main process exited, code=exited, status=143/n/a dec 20 14:16:34 systemd[1]: Stopped logstash. dec 20 14:16:34 systemd[1]: logstash.service: Unit entered failed state. dec 20 14:16:34 systemd[1]: logstash.service: Failed with result 'exit-code'. I temporary stopped logstash and trying to run it manually with debugging. ---
**Author:** ![Kristoffer\_Fagerlund](https://avatars.discourse-cdn.com/v4/letter/k/b3f665/32.png) [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Post date:** [December 20, 2017, 1:42pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/8 "2017-12-20T13:42:14Z")
The problem is solved if i manually start logstash with :/usr/share/logstash$ sudo bin/logstash --log.level=debug --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf how can I config the service to use these default paths? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 17, 2018, 1:42pm UTC](https://discuss.elastic.co/t/logstash-defaults-modules/112476/9 "2018-01-17T13:42:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.