hey there,
I am following the official document and seems like it's not working. I am using a freshly installed CentOS 7 system and have Java 1.8 installed. As per the document when i run below
[Foo@Elastic logstash]$ sudo bin/logstash -e 'input { stdin { } } output { stdout {} }'
It just stays like that no output. I have waited for 15 mins now. Any pointers why this is happening?
Thanks
V
Also after an long time i just typed Hello and below is what i have got
INFO: Logstash requires a setting file which is typically located in $LS_HOME/config or /etc/logstash. If you installed Logstash through a package and are starting it manually please specify the location to this settings file by passing in "--path.settings=/path/.."
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=Errno::ENOENT, message=>No such file or directory
Why's that ?
Well, as the document says - if you don't start logstash via the service, you should pass logstash.yml through the --path.settings i.e.
sudo bin/logstash --path.settings=/etc/logstash/logstash.yml -e 'input { stdin { } } output { stdout {} }'
Assuming you've installed it via the package.
Ah really ? but it's not even mention in the official document that i need to include --path.settings=/etc/logstash/logstash.yml. Few things in here worth checking
INFO: Logstash requires a setting file which is typically located in $LS_HOME/config or /etc/logstash. If you installed Logstash through a package and are starting it manually please specify the location to this settings file by passing in "--path.settings=/path/.."
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/etc/logstash/logstash.yml, exception=Errno::ENOENT, message=>No such file or directory - /etc/logstash/logstash.yml/logstash.yml
Regards
V
Hi,
I also have a similar problem following the official documentation, installing on an Amzon instance (CentOS 7).
The documentation is correct until "sudo yum install logstash".
After the command chkconfig --add logstash the documentation is not correct cause you need to be root to add it at boot so there's a sudo missing.
Even with sudo this command is not working the logstash bin is not found.
I expect that is the problem so that why Vishal_Sharma1 can't start it as service and try to run the bin.
Somebody can confirm that the yum installation is not working, can it be a rpm problem?
Cause I only copy paste the command and stop after the chkconfig --add logstash.
I checked in the /etc/init.d there's no logstash.
Thanks for any help.
Huh, it seems like you actually need to give it the directory, not the file itself (No such file or directory - /etc/logstash/logstash.yml/logstash.yml)
Try changing it to --path.settings=/etc/logstash
It does seem the documentation is not 100% up to date - You can see the changes (and where they say you should specify --path.settings) at the breaking changes page
ok let me clear few things in here
so after running for 15 mins i got below
sudo bin/logstash --path.settings=/etc/logstash -e 'input { stdin { } } output { stdout {} }'
[sudo] password for vishne0:
Hell0 !
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties.
2016-11-01 12:39:45,782 Api Webserver ERROR No log4j2 configuration file found. Using default configuration: logging only errors to the console.
[vishne0@Elastic logstash]$ Hell0 !
-bash: Hell0: command not found
In /var/log/logstash there is a file logstash-plain.log, it says below
[2016-11-01T12:39:45,739][ERROR][logstash.agent ] failed to fetch pipeline configuration {:message=>"No config files found: /etc/logstash/conf.d/*\nCan you make sure this path is a logstash config file?"}
now why it's looking at conf.d ? something is really really wrong in here.
We have an issue for that: https://github.com/elastic/logstash/issues/6062
We haven't figured out a way around it yet. It has to do with the fact that -e syntax was originally meant to be merged with the content of whatever config file you specified with -f so you could hypothetically have -e 'output { stdout { codec => rubydebug } } and just add stdout logging.
Now that configuration files and paths are specified in /etc/logstash/logstash.yml, and the packaged version specifies /etc/logstash/conf.d/*, that's why it's not working the way it did in older versions. We're still working on a resolution for this.
Ran into the exact same issue. With regards to Logstash only so far, the current directions/tutorial/setup guide is not very conducive to noobs like myself.
https://www.elastic.co/guide/en/logstash/5.0/installing-logstash.html
Here I tried copying a few config files around but it didn't help
I was trying version 5.0 out yesterday and came across a similar issue.
5.0 forces you to have the settings file. But the default settings file has a row:
path.config: /etc/logstash/conf.d
This becomes a problem if you run the application and use -f to point it to a custom configuration file like this.
sudo /usr/share/logstash/bin/logstash -f /home/ec2-user/logstashfileinput.yaml --path.settings /etc/logstash/
Whatever file you point to using -f gets ignored and instead uses the file located at /etc/logstash/conf.d/. Assuming there even is one. The settings file takes precedence over whatever you pass in as a command line flag. If you were already planning on using that file then it isn't a problem, but for my cases I wasn't.
I haven't tested it, but it's possible that whatever is in the settings file would also take precedence over -w and -b too.
It probably won't affect anything on our production machines, but it's a bit annoying when I am trying to quickly test different configs.
Let me ask a stupid question here since I am very green here. The config file that Logstash is wanting is really the yml for inputs/filters/outputs... right?
Correct.
There are now two "config" files.
The one that has been around forever is what holds the inputs, filters, and outputs. That is referenced by the -f command line flag.
The settings file introduced in 5.0 holds settings such as number of pipeline workers, batch size, config location, logging location, logging level, and other bits of info. The issue here is that these settings may exists as part of a command line flag, and also in the settings file.
Now should teh default install have created a default config/yml file? Both locations on my system were empty.
We have a potential fix to allow -e to still be used: create an empty file in /etc/logstash/conf.d. Call it whatever you like, e.g.
sudo touch /etc/logstash/conf.d/empty.conf
Even though the file is empty, it should prevent this error:
Should this work for those trying to use -e to do the test config in the docs, we will put an empty config file in /etc/logstash/conf.d in future package releases.
It's right here in the breaking changes documentation.
Yes, it does create a default in /etc/logstash/logstash.yml. If that's empty, then something is wrong. The package provides a file filled with appropriate default values. Something or someone may have overwritten this file.
However, there is no default pipeline configuration in /etc/logstash/conf.d.
Scratch that. We may try a different approach. We're discussing it now.
Ok gotcha, yes I have the default yml then. Sorry too much cmdline today 
So I copied the logstash.yml and the log4j2.properties file into /usr/share/logstash/config just to make sure both possible config places were populated. Still the error persists.
@skeer, the --path.settings flag is still necessary if you're running from the command line. If it is omitted, it will fail. That's the error message you're seeing.
Try:
sudo -u logstash /usr/share/logstash/bin/logstash \
--path.settings /etc/logstash \
-f /path/to/first-pipeline.conf \
--config.test_and_exit
I put it on multiple lines so you can see it easier. --path.settings is required, but in this case, -f overrides the --path.config in the settings file.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.