Logstash No Longer Accepting Lumberjack Input

Hey folks,

I'm running into an odd issue. I recently upgraded elasticsearch from 1.7 to 2.2.0, everything seemed to work fine. Yesterday my ELK server ran out of disk space, i shut it down, added a larger disk, started back up, and it started fine, but the lumberjack input stopped accepting any files. I have an s3 and collectd input, both working, but not lumberjack. No idea what's going on.

This is an EC2 instance, not running clustered since my 2nd node isn't discovering the primary (another issue for another time). Here is my config. I've removed the filters and restarted logstash, but still no go.

Also, I did double check from my instances that I can telnet to the configured server on port 5000, and since it stopped accepting logs from my dozen or so servers all at once, I'm fairly certain logstash-forwarder is not the issue. No other infrastructure changes have been made. Also, NO log activity is being recorded, nothing in logstash.log or logstash.err.

Elasticsearch 2.2.0
Logstash 1.5.4

input {
lumberjack {
codec => multiline {
pattern => "^\s"
what => "previous"
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
s3 {
type => "sureflight_elb"
bucket => "skyward-elb-logs"
credentials => ["key-redacted", "skey-redacted"]
region_endpoint => "us-west-2"
udp {
port => 25826
buffer_size => 1452
codec => collectd { }
type => "collectd"

output {
elasticsearch {
host => "localhost"
protocol => "http"
port => "9200"
cluster => "elasticsearch"

Ok I finally figured this out. Sharing for future use. Once I realized how to get logstash-forwarder to not run in silent mode it was obvious.

/opt/logstash-forwarder/bin/logstash-forwarder.sh -config /etc/logstash-forwarder -spool-size 1 -quiet=false -syslog=false


2016/02/17 16:48:31.757041 Failed to tls handshake with x509: cannot validate certificate for because it doesn't contain any IP SANs

When I restarted I knew that the public dns had changed on the server, what I didn't know was that the original certificate was created using that public dns. I created a new record in Route 53 for a static DNS and recreated the cert and distributed it. Logs are rolling in.