# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.0
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.1-1
Logstash has a new settings file which defines start up time settings. This file is typically located in $LS_HOME/config or /etc/logstash. If you installed Logstash through a package and are starting it manually please specify the location to this settings file by passing in "--path.settings=/path/.." in the command line options {:level=>:warn}
Failed to load settings file from "path.settings". Aborting... {"path.settings"=>"/usr/share/logstash/config", "exception"=>Errno::ENOENT, "message"=>"No such file or directory - /usr/share/logstash/config/logstash.yml", :level=>:fatal}
JAVA #java -version
openjdk version "1.8.0_101"
Logstash version
#yum list installed | grep logstash
logstash.noarch 1:5.0.0~alpha5-1 @logstash-5.0
Plugin beats
# /usr/share/logstash/bin/logstash-plugin list | grep beat
logstash-input-beats
The error message you pasted explains what went wrong and how to fix it. I'll re-paste what you have above and add line breaks to make it easier to read without scrolling left to right (I'm not trying to be patronizing. I'm trying to make it easier for future readers to see the issue and understand the fix):
Logstash has a new settings file which defines start up time settings.
This file is typically located in $LS_HOME/config or /etc/logstash.
If you installed Logstash through a package and are starting it manually
please specify the location to this settings file by passing in
"--path.settings=/path/.." in the command line options {:level=>:warn}
Failed to load settings file from "path.settings".
Aborting... {"path.settings"=>"/usr/share/logstash/config",
"exception"=>Errno::ENOENT,
"message"=>"No such file or directory - /usr/share/logstash/config/logstash.yml",
:level=>:fatal}
What this means is that you should start Logstash with a manually configured --path.settings, like this:
What version of RHEL/CentOS are you using? That may determine how Logstash needs to be started.
Did you edit /etc/logstash/logstash.yml? You shouldn't have needed to change the path.settings attribute as it was preconfigured to point there already.
Yes, i haved put the path.settings attribute after you say to run with this attribute, i try put in the logstash.yml configurations file, thinking it would work, but no.
That's why it isn't working. You're using upstart syntax and CentOS 7 is systemctl. Logstash 5 is auto-detecting the system init type and creating startup files accordingly. It just so happens that systemctl supersedes upstart, so it's the default on RHEL/CentOS 7.
In this way logstash really runs, but apparently he ignores the configuration file in the /etc/logstash/conf.d/logstash.conf , because the configured port for the beats don't up.
If i run in this way /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf, the port up.
Without some -v verbose logs (or possibly debug logs), I won't be able to know what's going on. You edited the default files, so I can't say for sure whether that caused it or not. It should have picked up anything ending in .conf in /etc/logstash/conf.d. This is being used by many people everywhere right now, so I'm keen to know why it's not working for you.
I only have one .conf file in the /etc/logstash/conf.d.
The -v option don't work in logstash. If you want reproduce, you can install CentOS (install java by yum) and install the logstash alpha5 by the yum repo.
[vagrant@localhost logstash]$ systemctl status logstash
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: disabled)
Active: active (running) since Fri 2016-08-19 17:31:57 UTC; 4min 1s ago
Main PID: 30241 (java)
CGroup: /system.slice/logstash.service
└─30241 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xm...
Aug 19 17:31:57 localhost.localdomain systemd[1]: Started logstash.
Aug 19 17:31:57 localhost.localdomain systemd[1]: Starting logstash...
Aug 19 17:32:23 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.0
Aug 19 17:32:23 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.1-1
Aug 19 17:32:28 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 17:32:28 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 17:32:28 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 17:32:28 localhost.localdomain logstash[30241]: --- jar coordinate com.fasterxml.jackson.module:jackson-module-afterburner already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 17:32:29 localhost.localdomain logstash[30241]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Aug 19 17:32:31 localhost.localdomain logstash[30241]: {:timestamp=>"2016-08-19T17:32:31.468000+0000", :message=>"Pipeline main started"}
As you can see, the only thing I've changed is the Logstash output (no elasticsearch, just stdout). I get the correct port running for Beats.
Again, I don't know what's going on inside your box, but I can't reproduce your problem locally
Hey Aaron, i think which the problem is something with permission. I saw that you were running with a non root user, and i intall all whith root. Then i restored a snapshot from the VM and add a new user(non root) and install all using sudo(e.g sudo yum intall logstash), thus it functioned.
If you can try install all with a root user, i think which you can reproduce.
The commands needed with root or sudo were anything yum, e.g. sudo yum install logstash, and systemctl, e.g. sudo systemctl start logstash. And of course, to edit the config file. I just used sudo, nothing special.
So, i understood what you said! What I said was that, when i installed, there was only the root user on the host. In this way it was installed, runned and edited all with the root user.
I believe that in this way you can reproduce, instaling and running all with root user, without using sudo(e.g # yum install logstash or # systemctl start logstash).
That shouldn't matter. To leave no stone unturned, I tried what you described. It still worked for me, however, it took longer to finish launching:
[root@localhost logstash]# systemctl status logstash
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: disabled)
Active: active (running) since Fri 2016-08-19 19:21:14 UTC; 4min 10s ago
Main PID: 30861 (java)
CGroup: /system.slice/logstash.service
└─30861 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xm...
Aug 19 19:21:14 localhost.localdomain systemd[1]: Started logstash.
Aug 19 19:21:14 localhost.localdomain systemd[1]: Starting logstash...
Aug 19 19:21:39 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.0
Aug 19 19:21:39 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.1-1
Aug 19 19:22:19 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 19:22:19 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 19:22:19 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 19:22:19 localhost.localdomain logstash[30861]: --- jar coordinate com.fasterxml.jackson.module:jackson-module-afterburner already loaded with version 2.7.1 - omit version 2.7.5
Aug 19 19:22:20 localhost.localdomain logstash[30861]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Aug 19 19:22:21 localhost.localdomain logstash[30861]: {:timestamp=>"2016-08-19T19:22:21.254000+0000", :message=>"Pipeline main started"}
When launched via sudo (see my earlier post), it took 26 seconds after launch before the jar messages start, and then another 8 seconds before Pipeline main started. When launched as root, it took 25 seconds after launch before the jar messages start, and then an additional 42 seconds before Pipeline main started.
I do not think this has anything to do with Logstash, as Logstash has already been started as the Logstash user, and both implementations seem to have reached the jar loading section at the same time, so permissions have already been set. I could be wrong, but I do not know what could be done by Logstash differently. It's not like there's a flag somewhere that detected the user which started the systemctl process that spawned the Logstash process.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.