Logstash 5 not running

@skeer, the --path.settings flag is still necessary if you're running from the command line. If it is omitted, it will fail. That's the error message you're seeing.

Try:

sudo -u logstash /usr/share/logstash/bin/logstash \
 --path.settings /etc/logstash \
 -f /path/to/first-pipeline.conf \
 --config.test_and_exit

I put it on multiple lines so you can see it easier. --path.settings is required, but in this case, -f overrides the --path.config in the settings file.

1 Like

Ok that sounds good.. I'd suggest that be mentioned specifically on the page: https://www.elastic.co/guide/en/logstash/5.0/first-event.html

Any other ways of solving this problem? I am still getting the same info error. I have done the following so far.
created a empty.conf file in /etc/logstash.conf.d &

sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /path/to/first-pipeline.conf --config.test_and_exit

I successfully configured it with the following response.

Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties.

I am sure you all are tired of this problem. Thanks for working with all of us on this.

so do we have any solution for this issue?
Regards
V

Hi all, I try to install ELK 5 in parallel with ELK ~2.4 and I encounter the same problem ...
I made a YUM install on centOS 7.2.
After reader last comments I have tried with command line
_sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.test.conf --config.test_and_exit

and every time logstash stopped...
With minima conf:

input { stdin { } }
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

Logs (/usr/share/logstash/logs/logstash-plain.log) are:
[2016-11-03T12:11:33,821][ERROR][logstash.agent ] fetched an invalid config {:config=>"logstash.test.confinput { stdin { type => stdin } }output { stdout { codec => rubydebug } }", :reason=>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "}

without file ...
[2016-11-03T12:12:26,247][FATAL][logstash.runner ] The given configuration is invalid. Reason: Expected one of #, input, filter, output at line 2, column 1 (byte 2) after
It can imagine my config or it's reading an other file ... ?

With the deamon:
CMD: journalctl -f --unit logstash
Nothing write on /usr/share/logstash/logs/logstash-plain.log

Nov 03 12:57:05 ELKv5 logstash[64485]: Sending Logstash logs to /usr/share/logstash/logs wh ich is now configured via log4j2.properties.
Nov 03 12:57:05 ELKv5 logstash[64485]: ERROR: No configuration file was specified. Perhaps you forgot to provide the '-f yourlogstash.conf' flag?
Nov 03 12:57:05 ELKv5 logstash[64485]: usage:
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -e CONFIG_STR [-t] [--log.level fatal|e rror|warn|info|debug|trace] [-w COUNT] [-l LOG]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -i SHELL [--log.level fatal|error|warn| info|debug|trace]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -V [--log.level fatal|error|warn|info|d ebug|trace]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash --help
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service: main process exited, code=exited, statu s=1/FAILURE
Nov 03 12:57:05 ELKv5 systemd[1]: Unit logstash.service entered failed state.
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service failed.
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service holdoff time over, scheduling restart.

Need help please

The easiest solution, is probably download the zip file and start log stash from there. But you could try to use the initctl command:

sudo initctl start logstash
ps aux | grep logstash

Not sure if the command will work for cento OS though it did for Ubuntu Trusty.
Also refer here: Logstash requires setting a file

For RAPY's conf, I believe that I had a similar issue earlier. I think I removed { codec => rubydebug } to fix this. See if this works for you.

Hi Chuen Lee, I try but same error.
As workaround I do this:

  • mkdir /usr/share/logstash/config
  • chown -R logstash: /usr/share/logstash/config
  • ln -s /etc/logstash/logstash.yml /usr/share/logstash/config/logstash.yml

then I lunch via command line ...
/usr/share/logstash/bin/logstash --verbose --debug --path.settings=/etc/logstash -f /etc/logstash/conf.d/

It work but but creating this directory /etc/logstash/${sys:ls.logs}

I hope This can help us to help me :smiley_cat:
I will try with tar.gz but my first goal is to use "classic install" if we want use it out of the POC.

Just wondering if we have any solution ?

My ES has not created any indices yet

I have upgraded logstash to version 5.0 (and elasticsearch of course) but since then I am not able to start logstash at all. It doesn't matter if I use the option -e or -f. I have even copied the config file to /etc/logstash/conf.d but this didn't help. When I start logstash with the following command sudo /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ -f path_to_config_file I don't get any output. Nothing is indexed in ES and I don't get any logs in /var/log/logstash although I've set the log leve to trace. Apart from that I have default configurations in the logstash.yml file (in /etc/logstash). I have installed logstash through the deb package but I have previously also tried with apt-get.
The only command that worked after upgrade is sudo /usr/share/logstash/bin/logstash --version which returns logstash 5.0.0

well Elastic 5 using yum repo on CentOS 7 working fine for me. Please check the log in /var/log/logstash
/logstash-plain.log ?

I'm checking exactly that file but I cannot see any logs there. I'm running ES and logstash on Ubuntu 14.04.

Please check /var/log/messages. It must be there

1 Like

I don't have a directory /var/log/messages. In the .yml settings file of logstash I have the following line path.logs: /var/log/logstash so I assume that the logs should be written to some file in this path.

No please check in /var/log/messages

Try the upstart command as shown here: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/deb.html
than use

ps aux | grep logstash

Paste your result here. And yes what does the directory:

/var/log/logstash

have in it?

In a Deb package, logs are stored in /var/log/logstash

Which upstart command do you mean? The provided link is about installing elasticsearch from the debian package. I have no problems with elasticsearch since the service is running (when I input sudo service elasticsearch status I get * elasticsearch is running.
in /var/log/logstash I have a file called logstash-plain.log but it doesn't contain any logs from today.

So now I could suddenly see thousands of logs. After more than half an hour from running the command sudo /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ -f path_to_config_file it started running and I could suddenly see tens of thousands of logs in the log file (for a time window of almost one hour, so all logs of the same hour popped up suddenly in the file). Now i changed the log level back to default (by commenting the line in .yml file) and started logstash with the same command again and I'm still waiting for the output.
However I hope it is not normal to wait for more than 30 minutes for logstash to start running and for the logs to get written so do you know where I can start looking for the root cause of this problem?