Logstash requires setting a file

Logstash Info error: A setting file is missing
I installed the deb file of logstash 5.0 and I starting logstash for the first time. I get an INFO error:

INFO: Logstash requires a setting file which is typically located in $LS_HOME/config or /etc/logstash. If you installed Logstash through a package and are starting it manually please specify the location to this settings file by passing in "--path.settings=/path/.."
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=Errno::ENOENT, message=>No such file or directory - /usr/share/logstash/config/logstash.yml

This seems pretty obvious to fix. So I created a directory LS_HOME/config and moved my logstash-simple.conf file there and my logstash.yml from /etc/logstash/conf.d to /etc/logstash/ directory Same INFO error

So I than used the -f flag to specified the location of the configuration file. Same INFO error. So it seems like /bin/logstash script is trying to look for a setting file .config or logstash.yml even though both files exist in the request places.

When I use:

bin/logstash -e 'input { stdin { } } output { stdout {} }'

same INFO error appears. I am out ideas. Does anyone know how to fix this INFO error?
*This is a fresh install from the logstash 5.0 deb file.
Documentation:https://www.elastic.co/guide/en/logstash/5.0/first-event.html

Sorry for the confusion. We are making changes to the -e flag as soon as we can. It behaves differently for package (DEB/RPM) installed versions than the tarball installed version because Logstash needs --path.settings to be set to run, and packages install the correct one in /etc/logstash. The assumption made was that people installing with packages are going to go straight to /etc/logstash/conf.d to put their pipeline configuration, rather than experiment with the -e flag.

As far as $LS_HOME is concerned, package installed variants are again, different. $LS_HOME doesn't matter because it looks in /etc/logstash/logstash.yml to find all of the settings, rather than $LS_HOME/config. Putting config there is just going to cause future difficulty.

Thanks for the quick response ! I just messaged you on another post as well.

The reason for all of this headache is that we assumed people would be starting Logstash via upstart and systemd, not at the command-line. We have over 5 years of that use pattern that doesn't just go away. 5.0 is a major release, so we figured we could make some breaking changes and that would be acceptable. These are the growing pains :slight_smile:

2 Likes

For anyone else coming here:

cd logstash-5.0.0
bin/logstash -e 'input { stdin { } } output { stdout {} }'

This step from the documentation only works this way if you install Logstash as a tarball, not as an RPM or DEB package. This is because the `$LS_HOME` variable is set locally, rather than finding configuration settings in `/etc/logstash`, which is what happens with a package install.

The `-e` flag was designed to merge your command-line configuration with whatever was in the `-f` (now `--path.config`) pipeline configuration file.  If `-f` was omitted, then `-e` was the _only_ configuration.  With `--path.settings` providing a default `--path.config` of `/etc/logstash/conf.d`, if there are no files there, you get a glob error that states no file was found, and things don't work.  We're going to change the `-e` and `-f` flags to be mutually exclusive in a soon-to-come patch release.  https://github.com/elastic/logstash/issues/6164 and https://github.com/elastic/logstash/issues/6165
1 Like

If you are using linux:

Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

The upstart command works but the error will still persist if you directly query with logstash:

sudo initctl start logstash

Which error? The one that says you need to have a configuration file in /etc/logstash/conf.d, or the one about logstash.yml? I ask because out of the box, if you start with a pipeline configuration in /etc/logstash/conf.d using the created startup script, you should not see the No such file or directory - /usr/share/logstash/config/logstash.yml error because the packaged version's logstash.yml file is in /etc/logstash, and the startup scripts are configured with this.

Yes it was the same error but works now. Not sure what happened last night.

I am kind of goin nuts now, lol. What good be the correct command line to run with falgs included for running Logstash 5? I have tried many things but keep getting errors such as [2017-02-16T15:10:12,995][INFO ][logstash.agent ] No config files found in path {:path=>"/opt/logstash/logstash-db-shipper.conf"} [2017-02-16T15:10:13,000][ERROR][logstash.agent ] failed to fetch pipeline configuration {:message=>"No config files found: /opt/logstash/logstash-db-shipper.conf. Can you make sure this path is a logstash config file?"}. I'd really appreaciate anyone who can give me example please. :frowning:

If you are still determined to launch Logstash from the command-line (rather than sysv/upstart/systemd), then you need a --path.settings at the very least. By default, --path.settings will point to $LS_HOME/config, where $LS_HOME is the directory where Logstash is installed. However, if you installed Logstash via an RPM or DEB, then --path.settings defaults to /etc/logstash. In order to make Logstash work at the command-line, you need to manually specify the --path.settings to the correct location. Inside $LS_HOME/config or /etc/logstash is the logstash.yml file, and a few others. This should be edited accordingly. You can override pretty much anything in the logstash.yml file with command-line flags, such as --path.config (which is the same thing as -f), which should point to a configuration file or directory of config files.

2 Likes

Thanks! I found my issue. It was a cronjob that was an outdated elasticsearch installation. Thanks again. :slight_smile: