No output to Elasticsearch

Elastic Stack Logstash
1

Hi,

I hope somebody can assist?

I have setup Elasticsearch, Kibana, Logstash and Filebeat but can't get Logstash to write to Elasticsearch.

Filebeat logs to Logstash. I know that because in my Filebeat log file I get these entries:

INFO Events sent: 2048
2016-04-15T12:39:28Z INFO Registry file updated. 100 states written.

Kibana can access Elasticsearch.It complains that it is unable to fetch mapping becuase it can't find anything logstash related under "/var/lib/elasticsearch/elasticsearch/nodes/0/indices/". ther is just a .kibana directory.

The logstash log file only has one entry upon start-up: ":message=>"Pipeline main started"}
"

The kibana/elastic/logstash/filebeat log files doesn't have any error.

I use these versions:
Logstash 2.3.1
Elasticsearch 2.3.1
Kibana 4.4.2
Filebeat 1.2.1

Logstash config:

input {
beats{
port => 5044 # number (required)
codec => json_lines {
charset => "UTF-8"
}
}
}

filter {

    if ("New client unix socket" in [message])  {
            drop { }
    }

    if ("Client unix socket" in [message])  {
            drop { }
    }

    if ("logstash_heartbeat" in [message])  {
          drop { }
    }

    mutate {
            gsub => ["type", "[.]","_"]
    }

    metrics {
            meter => [ "%{system}.%{type}.%{logLevel}"]
            clear_interval => 300
            flush_interval => 300
            add_tag => [ "perlog" ]
            percentiles => [1]
    }

}

output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

I am not sure what I am doing wrong. Please help!

2

In logstash output section, you can turn on debug mode to output to stdout

output {
    stdout { codec => rubydebug  }
}

with this you can see if Logstash processes anything or gets any error.