Ok that sounds good.. I'd suggest that be mentioned specifically on the page: https://www.elastic.co/guide/en/logstash/5.0/first-event.html
Any other ways of solving this problem? I am still getting the same info error. I have done the following so far.
created a empty.conf file in /etc/logstash.conf.d &
sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /path/to/first-pipeline.conf --config.test_and_exit
I successfully configured it with the following response.
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties.
I am sure you all are tired of this problem. Thanks for working with all of us on this.
so do we have any solution for this issue?
Regards
V
Hi all, I try to install ELK 5 in parallel with ELK ~2.4 and I encounter the same problem ...
I made a YUM install on centOS 7.2.
After reader last comments I have tried with command line
_sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.test.conf --config.test_and_exit
and every time logstash stopped...
With minima conf:
input { stdin { } }
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
Logs (/usr/share/logstash/logs/logstash-plain.log) are:
[2016-11-03T12:11:33,821][ERROR][logstash.agent ] fetched an invalid config {:config=>"logstash.test.confinput { stdin { type => stdin } }output { stdout { codec => rubydebug } }", :reason=>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "}
without file ...
[2016-11-03T12:12:26,247][FATAL][logstash.runner ] The given configuration is invalid. Reason: Expected one of #, input, filter, output at line 2, column 1 (byte 2) after
It can imagine my config or it's reading an other file ... ?
With the deamon:
CMD: journalctl -f --unit logstash
Nothing write on /usr/share/logstash/logs/logstash-plain.log
Nov 03 12:57:05 ELKv5 logstash[64485]: Sending Logstash logs to /usr/share/logstash/logs wh ich is now configured via log4j2.properties.
Nov 03 12:57:05 ELKv5 logstash[64485]: ERROR: No configuration file was specified. Perhaps you forgot to provide the '-f yourlogstash.conf' flag?
Nov 03 12:57:05 ELKv5 logstash[64485]: usage:
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -e CONFIG_STR [-t] [--log.level fatal|e rror|warn|info|debug|trace] [-w COUNT] [-l LOG]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -i SHELL [--log.level fatal|error|warn| info|debug|trace]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash -V [--log.level fatal|error|warn|info|d ebug|trace]
Nov 03 12:57:05 ELKv5 logstash[64485]: bin/logstash --help
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service: main process exited, code=exited, statu s=1/FAILURE
Nov 03 12:57:05 ELKv5 systemd[1]: Unit logstash.service entered failed state.
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service failed.
Nov 03 12:57:05 ELKv5 systemd[1]: logstash.service holdoff time over, scheduling restart.
Need help please
The easiest solution, is probably download the zip file and start log stash from there. But you could try to use the initctl command:
sudo initctl start logstash
ps aux | grep logstash
Not sure if the command will work for cento OS though it did for Ubuntu Trusty.
Also refer here: Logstash requires setting a file - #4 by theuntergeek
For RAPY's conf, I believe that I had a similar issue earlier. I think I removed { codec => rubydebug } to fix this. See if this works for you.
Hi Chuen Lee, I try but same error.
As workaround I do this:
- mkdir /usr/share/logstash/config
- chown -R logstash: /usr/share/logstash/config
- ln -s /etc/logstash/logstash.yml /usr/share/logstash/config/logstash.yml
then I lunch via command line ...
/usr/share/logstash/bin/logstash --verbose --debug --path.settings=/etc/logstash -f /etc/logstash/conf.d/
It work but but creating this directory /etc/logstash/${sys:ls.logs}
I hope This can help us to help me 
I will try with tar.gz but my first goal is to use "classic install" if we want use it out of the POC.
Just wondering if we have any solution ?
My ES has not created any indices yet
I have upgraded logstash to version 5.0 (and elasticsearch of course) but since then I am not able to start logstash at all. It doesn't matter if I use the option -e or -f. I have even copied the config file to /etc/logstash/conf.d but this didn't help. When I start logstash with the following command sudo /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ -f path_to_config_file I don't get any output. Nothing is indexed in ES and I don't get any logs in /var/log/logstash although I've set the log leve to trace. Apart from that I have default configurations in the logstash.yml file (in /etc/logstash). I have installed logstash through the deb package but I have previously also tried with apt-get.
The only command that worked after upgrade is sudo /usr/share/logstash/bin/logstash --version which returns logstash 5.0.0
well Elastic 5 using yum repo on CentOS 7 working fine for me. Please check the log in /var/log/logstash
/logstash-plain.log ?
I'm checking exactly that file but I cannot see any logs there. I'm running ES and logstash on Ubuntu 14.04.
Please check /var/log/messages. It must be there
1 Like
I don't have a directory /var/log/messages. In the .yml settings file of logstash I have the following line path.logs: /var/log/logstash so I assume that the logs should be written to some file in this path.
No please check in /var/log/messages
Try the upstart command as shown here: Install Elasticsearch with Debian Package | Elasticsearch Guide [5.0] | Elastic
than use
ps aux | grep logstash
Paste your result here. And yes what does the directory:
/var/log/logstash
have in it?
In a Deb package, logs are stored in /var/log/logstash
Which upstart command do you mean? The provided link is about installing elasticsearch from the debian package. I have no problems with elasticsearch since the service is running (when I input sudo service elasticsearch status I get * elasticsearch is running.
in /var/log/logstash I have a file called logstash-plain.log but it doesn't contain any logs from today.
So now I could suddenly see thousands of logs. After more than half an hour from running the command sudo /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ -f path_to_config_file it started running and I could suddenly see tens of thousands of logs in the log file (for a time window of almost one hour, so all logs of the same hour popped up suddenly in the file). Now i changed the log level back to default (by commenting the line in .yml file) and started logstash with the same command again and I'm still waiting for the output.
However I hope it is not normal to wait for more than 30 minutes for logstash to start running and for the logs to get written so do you know where I can start looking for the root cause of this problem?
My apologizes, here is the correct link: https://www.elastic.co/guide/en/logstash/5.0/running-logstash.html#running-logstash-upstart
Which line in the logstash.yml file did you comment out? And it sounds like a performance issue. What is your system configuration? CPU? RAM? Disk space?
Some fun facts on performance issues with logstash: https://www.elastic.co/guide/en/logstash/current/performance-troubleshooting.html