# Logstash doesn't start

**URL:** <https://discuss.elastic.co/t/logstash-doesnt-start/236919>\
**Category:** Logstash\
**Created:** [June 12, 2020, 4:05pm UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919 "2020-06-12T16:05:16Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 8:32am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/21 "2020-06-18T08:32:20Z")

</div>

> [2020-06-18T10:31:56,332][DEBUG][logstash.runner] --------------- Logstash Settings -------------------  
> [2020-06-18T10:31:56,424][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=\>"/etc/logstash/pipelines.yml"}  
> [2020-06-18T10:31:56,484][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 8:38am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/22 "2020-06-18T08:38:09Z")

</div>

If I set path.config: "/etc/logstash/conf.d/01-wazuh.conf" in logstash.yml this is output:

> [2020-06-18T10:36:35,235][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
> [2020-06-18T10:36:35,246][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.7.1"}  
> [2020-06-18T10:36:35,338][DEBUG][logstash.agent] Setting up metric collection  
> [2020-06-18T10:36:35,475][DEBUG][logstash.instrument.periodicpoller.os] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T10:36:35,958][DEBUG][logstash.instrument.periodicpoller.jvm] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T10:36:36,210][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}  
> [2020-06-18T10:36:36,220][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 8:41am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/23 "2020-06-18T08:41:23Z")

</div>

> [@Uzzi](#):
>
> [2020-06-18T10:36:35,235][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
> [2020-06-18T10:36:35,246][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.7.1"}  
> [2020-06-18T10:36:35,338][DEBUG][logstash.agent] Setting up metric collection  
> [2020-06-18T10:36:35,475][DEBUG][logstash.instrument.periodicpoller.os] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T10:36:35,958][DEBUG][logstash.instrument.periodicpoller.jvm] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T10:36:36,210][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}  
> [2020-06-18T10:36:36,220][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}

This is the expected behavior, it means no error and your logstash work as expected,

I saw that you run with this command

```auto
/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash --config.debug

```

It should be like the last command:

```auto
/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash/conf.d/ --config.debug

```

After all set, please remove the debug option. Use systemd start/stop of logstash, then check use systemctl status logstash.

Regards,  
Fadjar Tandabawana

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 8:49am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/24 "2020-06-18T08:49:20Z")

</div>

/usr/share/logstash/bin/logstash --config.test\_and\_exit --path.settings /etc/logstash --config.debug---\>  
Using config.test\_and\_exit mode. Config Validation Result: OK. Exiting Logstash

/usr/share/logstash/bin/logstash --config.test\_and\_exit --path.settings /etc/logstash/conf.d/ --config.debug----\>  
ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/conf.d/pipelines.yml  
usage

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 8:52am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/25 "2020-06-18T08:52:02Z")

</div>

Sorry, my mistake  
You right.. the first to check the config.

So, what is the problem for your config?

The result of the log of the logstash is the expected behavior until there are explicit ERROR message occurred.

Regards,  
Fadjar Tandabawana

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 8:55am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/26 "2020-06-18T08:55:08Z")

</div>

@fadjar340 I'm bercome crazy!!!!

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/01-wazuh.conf works fine!!!!

/usr/share/logstash/bin/logstash -f /etc/logstash/logstash.yml---\>

> [ERROR] 2020-06-18 10:56:49.875 [Converge PipelineAction::Create] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "input", "filter", "output" at line 19, column 1 (byte 320) after ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge\_state'"]}  
> [INFO] 2020-06-18 10:56:50.552 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 9:03am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/27 "2020-06-18T09:03:43Z")

</div>

This is definitely wrong and you got the ERROR message.

> [@Uzzi](#):
>
> /usr/share/logstash/bin/logstash -f /etc/logstash/logstash.yml---\>

Also

```auto
/usr/share/logstash/bin/logstash -f /etc/logstash/pipeline.yml <=wrong

```

The logstash.yml and pipeline.yml will read when the --path.settings .  
If you want to config the input, filter and output just use -f /etc/logstash/conf.d/

So, please don't use pipeline.yml as process configuration.  
Better, if you want to use default pipeline.id: main, just put into logstash.yml and put the process configuration into /etc/logstash/conf.d/ accordingly.

Regards,  
Fadjar Tandabawana

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 9:05am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/28 "2020-06-18T09:05:50Z")

</div>

This is my actual logstash.yml:  
pipeline.id: main  
path.config: "/etc/logstash/conf.d/"

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 9:12am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/29 "2020-06-18T09:12:01Z")

</div>

If you put the process config in the default folder /etc/logstash/conf.d/01-wazuh.conf ,  
just disable the path.config: "/etc/logstash/conf.d/", because the default logstash config for processing in that folder, instead you put into another folder, then you need to use pipeline.yml.

I wish your logstash works as expected...

Regards,  
Fadjar Tandabawana

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 9:15am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/30 "2020-06-18T09:15:11Z")

</div>

I've removed path.config than /usr/share/logstash/bin/logstash -f /etc/logstash/logstash.yml---\>

> [WARN] 2020-06-18 11:13:39.323 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified  
> [INFO] 2020-06-18 11:13:39.340 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=\>"7.7.1"}  
> [ERROR] 2020-06-18 11:13:41.110 [Converge PipelineAction::Create] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "input", "filter", "output" at line 19, column 1 (byte 320) after ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge\_state'"]}  
> [INFO] 2020-06-18 11:13:41.627 [Api Webserver] agent - Successfully started

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 9:19am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/31 "2020-06-18T09:19:49Z")

</div>

👍 👍 👍 👍

Try just run the logstash from systemctl..

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 9:23am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/32 "2020-06-18T09:23:12Z")

</div>

@fadjar340 This is complete /usr/share/logstash/bin/logstash -f /etc/logstash/logstash.yml output:

> root@elk1:~# /usr/share/logstash/bin/logstash -f /etc/logstash/logstash.yml  
> OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.  
> WARNING: An illegal reflective access operation has occurred  
> WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long)  
> WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules  
> WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations  
> WARNING: All illegal access operations will be denied in a future release  
> WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults  
> Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console  
> [WARN] 2020-06-18 11:13:39.323 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified  
> [INFO] 2020-06-18 11:13:39.340 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=\>"7.7.1"}  
> [ERROR] 2020-06-18 11:13:41.110 [Converge PipelineAction::Create] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "input", "filter", "output" at line 19, column 1 (byte 320) after ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge\_state'"]}  
> [INFO] 2020-06-18 11:13:41.627 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}  
> [INFO] 2020-06-18 11:13:46.433 [LogStash::Runner] runner - Logstash shut down.

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 9:25am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/33 "2020-06-18T09:25:30Z")

</div>

> [@Uzzi](#):
>
> pipeline.id: main  
> path.config: "/etc/logstash/conf.d/"

Try this in the logstash.yml

```auto
pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf"

```

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 9:29am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/34 "2020-06-18T09:29:00Z")

</div>

Same bad output. 🥵 😪  
I'm using logstash.version"=\>"7.7.1 on OpenJDK Runtime Environment (build 11.0.7+10-post-Debian-3deb10u1)

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [June 18, 2020, 9:45am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/35 "2020-06-18T09:45:02Z")

</div>

Did you following this:  
[https://documentation.wazuh.com/3.9/installation-guide/installing-elastic-stack/transform\_logstash.html](https://documentation.wazuh.com/3.9/installation-guide/installing-elastic-stack/transform_logstash.html)

I think you need to follow exact procedures....

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 9:46am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/36 "2020-06-18T09:46:42Z")

</div>

Yes I've followed [https://documentation.wazuh.com/3.12/installation-guide/installing-elastic-stack/transform\_logstash.html](https://documentation.wazuh.com/3.12/installation-guide/installing-elastic-stack/transform_logstash.html)

---

<div class="post-metadata">

**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\
**Post date:** [June 18, 2020, 10:46am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/37 "2020-06-18T10:46:08Z")

</div>

> [@Uzzi](#):
>
> logstash.yml: [https://paste.debian.net/1152630/](https://paste.debian.net/1152630/)  
> pipeline.yml: [https://paste.debian.net/1152631/](https://paste.debian.net/1152631/)

from your logstash.yml

```
# The ID of the pipeline.
#
pipeline.id: main

```

your logstash.yml has pipeline definition, with the id "main", and your pipelines.yml also has pipeline definition with id "main". I suggest just choose one of two.

there should be a default logstash.yml in /etc/logstash. you can backup your existing logstash.yml and restore the default logstash.yml in place.

never use logstash -f with logstash.yml. that file is logstash settings file, not logstash pipeline configuration file. read [here](https://www.elastic.co/guide/en/logstash/current/config-setting-files.html#config-setting-files) for the difference , as well as information for pipelines.yml

you only pass -f to pipeline configuration files (either files or directory). read [here](https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html) for various options in running Logstash from CLI.

Assuming you're not making any changes to systemd, and you're using deb / rpm system : when you run logstash (without any parameters) from CLI, or using systemctl, logstash will read settings. from /etc/logstash. there's no need to pass --path.settings options

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 11:09am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/38 "2020-06-18T11:09:32Z")

</div>

I've replaced logstash.yml and pipelines.yml with defaults  
[https://paste.debian.net/1152655/](https://paste.debian.net/1152655/)  
[https://paste.debian.net/1152657/](https://paste.debian.net/1152657/)

than systemctl start logstash ---\> same error

> [2020-06-18T13:03:22,291][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=\>"/etc/logstash/pipelines.yml"}  
> [2020-06-18T13:03:22,344][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.7.1"}  
> [2020-06-18T13:03:22,479][DEBUG][logstash.agent] Setting up metric collection  
> [2020-06-18T13:03:22,584][DEBUG][logstash.instrument.periodicpoller.os] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T13:03:23,058][DEBUG][logstash.instrument.periodicpoller.jvm] Starting {:polling\_interval=\>5, :polling\_timeout=\>120}  
> [2020-06-18T13:03:23,293][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ParNew"}  
> [2020-06-18T13:03:23,303][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=\>"ConcurrentMarkSweep"}

🥵

---

<div class="post-metadata">

**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\
**Post date:** [June 18, 2020, 11:59am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/39 "2020-06-18T11:59:27Z")

</div>

what does 01-wazuh.conf looks like ?

---

<div class="post-metadata">

**Author:** ![Uzzi](https://avatars.discourse-cdn.com/v4/letter/u/6a8cbe/32.png) [@Uzzi](https://discuss.elastic.co/u/Uzzi)\
**Post date:** [June 18, 2020, 12:28pm UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919/40 "2020-06-18T12:28:30Z")

</div>

curl -so /etc/logstash/conf.d/01-wazuh.conf [https://raw.githubusercontent.com/wazuh/wazuh/v3.9.5/extensions/logstash/7.x/01-wazuh-remote.conf](https://raw.githubusercontent.com/wazuh/wazuh/v3.9.5/extensions/logstash/7.x/01-wazuh-remote.conf)

[Previous page](https://discuss.elastic.co/t/logstash-doesnt-start/236919.md?page=1)

[Next page](https://discuss.elastic.co/t/logstash-doesnt-start/236919.md?page=3)
