Logstash service keeps crashing


(Simroo) #1

Hello
I have som problems with logstash and it keeps crashing.
Im not sure why, i thought it was a premisson problem but after chmod 777 on the files, the problem still prisets.
Note that this is during installation on a Debian Linux.

sudo service logstash start
logstash started.
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is running
@netmanager:/etc$ sudo service logstash status
logstash is not running
@netmanager:/etc$ sudo service logstash status
logstash is not running
@netmanager:/etc$ sudo service logstash status
logstash is not running
@netmanager:/etc$ sudo service logstash configtest
Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after.
There is nothing in the logs either :

@netmanager:/var/log/logstash$ ls
logstash.err logstash.log logstash.stdout
@netmanager:/var/log/logstash$ cat logstash.err
@netmanager:/var/log/logstash$ cat logstash.
logstash.err logstash.log logstash.stdout
@netmanager:/var/log/logstash$ cat logstash.stdout
Sending logstash logs to /var/log/logstash/logstash.log.
Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after
You may be interested in the '--configtest' flag which you can
use to validate logstash's configuration before you choose
to restart a running system.
srooprai@netmanager:/var/log/logstash$ cat logstash.log
{:timestamp=>"2015-12-15T17:58:24.344000+0100", :message=>"Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "}
{:timestamp=>"2015-12-15T17:58:24.351000+0100", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}
{:timestamp=>"2015-12-15T17:59:13.478000+0100", :message=>"Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "}
{:timestamp=>"2015-12-15T17:59:13.495000+0100", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}
{:timestamp=>"2015-12-15T18:02:15.340000+0100", :message=>"Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "}
{:timestamp=>"2015-12-15T18:02:15.347000+0100", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system."}

Here are my config files

:/etc/logstash$ cd conf.d/
@netmanager:/etc/logstash/conf.d$ ls
02-filebeat-input.conf 10-syslog.conf 30-elasticsearch-output.conf
@netmanager:/etc/logstash/conf.d$ cat 02-filebeat-input.conf
02-filebeat-input.conf

input {
beats {
port => 5044
type => "logs"
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

@netmanager:/etc/logstash/conf.d$ cat 10-syslog.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
@netmanager:/etc/logstash/conf.d$ cat 30-elasticsearch-output.conf
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

What iam i missing ?


(Magnus Bäck) #2

Does your 02-filebeat-input.conf actually begin with "02-filebeat-input.conf" as indicated above? If not, what if you follow Logstash's suggestion and run .../logstash --configtest -f filename.conf on each file in the conf.d directory? That should at least indicate which file Logstash doesn't like.


(Simroo) #3

Hello
The file name is "02-filebeat-input.conf". But im actully having trouble running the command that you sugested.
.../logstash --configtest -f filename.conf, i have tried .../logstash --configtest -f 02-filebeat-input.conf. Even as sudo.
Iam i doing this wrong ? Should i be in a spefic dir ?


(Magnus Bäck) #4

The file name is "02-filebeat-input.conf".

Yes. But this is what you pasted in your previous post:

$ cat 02-filebeat-input.conf
02-filebeat-input.conf

input {
beats {
port => 5044
type => "logs"
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

So, does the file actually begin with "02-filebeat-input.conf"?

i have tried .../logstash --configtest -f 02-filebeat-input.conf.

Replace "..." with the path to your Logstash binary.


(Simroo) #5

Hello
Sorry the file name is 02-filebeat-input.conf
/etc/logstash/conf.d$ ls
02-filebeat-input.conf


(Simroo) #6

@netmanager:/opt/logstash/bin$ /opt/logstash/bin/logstash --configtest -f 02-filebeat-input.conf.d
Error: No config files found: /opt/logstash/bin/02-filebeat-input.conf.d
Can you make sure this path is a logstash config file?
@netmanager:/opt/logstash/bin$ /opt/logstash/bin/logstash --configtest -f 10-syslog.conf
Error: No config files found: /opt/logstash/bin/10-syslog.conf
Can you make sure this path is a logstash config file?
@netmanager:/opt/logstash/bin$ /opt/logstash/bin/logstash --configtest -f 30-elasticsearch-output.conf
Error: No config files found: /opt/logstash/bin/30-elasticsearch-output.conf
Can you make sure this path is a logstash config file?

It says the same thing for all 3 files.... this is not good is it.


(Magnus Bäck) #7

Just use the absolute path to the files!

/opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/02-filebeat-input.conf

(Simroo) #8

IT works now :slight_smile:
$ /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/02-filebeat-input.conf
Error: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after
@netmanager:~$ /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/10-syslog.conf
Configuration OK
@netmanager:~$ /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/30-elasticsearch-output.conf
Configuration OK
@netmanager:~$ sudo nano /etc/logstash/conf.d/02-filebeat-input.conf
[sudo] password for :
@netmanager:~$ /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/02-filebeat-input.conf
Configuration OK
@netmanager:~$ sudo service logstash configtest
Configuration OK

There was a problem with the 02-filebeat-input.conf file. Not sure what but i repasted the config and saved again.
And it apears to be working again.
Thanks for all your help !
Perhaps i should stop using nano as a text editor and learn VIM


(system) #9