Hey folks,
I'm running into an odd issue. I recently upgraded elasticsearch from 1.7 to 2.2.0, everything seemed to work fine. Yesterday my ELK server ran out of disk space, i shut it down, added a larger disk, started back up, and it started fine, but the lumberjack input stopped accepting any files. I have an s3 and collectd input, both working, but not lumberjack. No idea what's going on.
This is an EC2 instance, not running clustered since my 2nd node isn't discovering the primary (another issue for another time). Here is my config. I've removed the filters and restarted logstash, but still no go.
Also, I did double check from my instances that I can telnet to the configured server on port 5000, and since it stopped accepting logs from my dozen or so servers all at once, I'm fairly certain logstash-forwarder is not the issue. No other infrastructure changes have been made. Also, NO log activity is being recorded, nothing in logstash.log or logstash.err.
Elasticsearch 2.2.0
Logstash 1.5.4
input {
lumberjack {
codec => multiline {
pattern => "^\s"
what => "previous"
}
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
s3 {
type => "sureflight_elb"
bucket => "skyward-elb-logs"
credentials => ["key-redacted", "skey-redacted"]
region_endpoint => "us-west-2"
}
udp {
port => 25826
buffer_size => 1452
codec => collectd { }
type => "collectd"
}
}
output {
elasticsearch {
host => "localhost"
protocol => "http"
port => "9200"
cluster => "elasticsearch"
}
}