i have performing basic setup of logstash getting JAVA error. not able to complete the config test.
Error:
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/etc/logstash/logstash.yml, exception=Java::JavaLang::RuntimeException, message=>Unhandled IOException: java.io.IOException: unhandled errno: Not a directory
[ERROR] 2018-03-26 01:42:49.137 [main] Logstash - java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
This is my startup.config file for logstash
Override Java location
#JAVACMD=/usr/lib/jvm/java
JAVA_HOME="/usr/lib/jvm/jre-1.8.0"
Set a home directory
LS_HOME=/etc/logstash
logstash settings directory, the path which contains logstash.yml
LS_SETTINGS_DIR=/etc/logstash
Arguments to pass to logstash
LS_OPTS="--path.settings ${LS_SETTINGS_DIR}"
Arguments to pass to java
LS_JAVA_OPTS=""
pidfiles aren't used the same way for upstart and systemd; this is for sysv users.
LS_PIDFILE=/var/run/logstash.pid
user and group id to be invoked as
LS_USER=logstash
LS_GROUP=logstash
Enable GC logging by uncommenting the appropriate lines in the GC logging
section in jvm.options
LS_GC_LOG_FILE=/var/log/logstash/gc.log
Open file limit
LS_OPEN_FILES=16384
Nice level
LS_NICE=19
"startup.options" 55L, 1696C
you set the wrong config file for parameter path.setting!
In that param you need to set input\output config for logstash
path.setting =/etc/logstash/conf.d/*.conf
and create in folder /etc/logstash/conf.d/ file logstash.conf with input\output config!
logstash.yml sets for logstash in 2 ways:
-
as a parameter commandline
https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html#command-line-flags
-
as a variable in startup.options file
/# logstash settings directory, the path which contains logstash.yml
LS_SETTINGS_DIR=/etc/logstash
do you mean .
i have to provide the logstash.config file path here
Arguments to pass to logstash
LS_OPTS="--path.settings ${LS_SETTINGS_DIR}"
because i have already created the conf.d folder and the .conf file is available in the folder
it works fine when i run the command /bin/logstash -f /etc/logstash/conf.d/logstash.conf
but logstash does not listen on port 5044 . so i looked at this error when i ran the command /usr/share/logstash/bin/logstash --path.settings /etc/logstash/logstash.yml -t
check rights for this config file with command ls -lh . -rw-rw---- for user logstash from group logstash.
(chown and chmod 660)
In some cases logstash cant read this file and goes down with errors
yea this is done. but i am not able to see logstash listening at 5044 port number.
any checks ???
i have tried alot making this logstash up . looking at basic structure to collect var log files from some 200 linux box machines
show your input and output logstash config, and show logstash log. If there any error on logstash start - it will be there.
for example i use winlogbeat and meticbeat for input, so i use:
logstash.conf
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["xx.xx.xx.xx:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
}
so, after restart logstash service, if everything configured right, i use netstat -tulpn and see that
tcp6 0 0 :::5044 :::* LISTEN
340/java
can i use filebeat instead of metric beat ?
i want to basically capture the syslogs and auth.log files details from all 200 machines
metricbeat is gathering system info about CPU, RAM, HDD, LAN and etc usage, and processes on PC. It doesnt collect or send any logs.
WInlogbeat collects and sends windows system logs (application, security, system, setup or others)
So, if you want to collect Windows syslogs - better use Winlogbeat (because it has filters and thin configuration).
If you need some other logs from different apps - use Filebeat - it was made for it.
thanks for the info . how do i connect to elastic search cloud setup using local logstash server