Hi.
I use ELK with Logstash forwarder.
Sometimes I have a lot of error in my logstash.log :
Here is my logstash configuration :
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/logstash/mycrt.crt"
ssl_key => "/etc/logstash/mykey.key"
codec => json {
charset => "Windows-1252"
}
}
lumberjack {
port => 5001
type => "logs"
ssl_certificate => "/etc/logstash/mycrt.crt"
ssl_key => "/etc/logstash/mykey.key"
codec => plain {
charset => "Windows-1252"
}
}
}
output {
if "grokparsefailure" in [tags] or "dateparsefailure" in [tags] {
file {
path => "/var/log/logstash/parsefailure.log"
}
}
elasticsearch {
hosts => ["myhost:9200"]
sniffing => true
manage_template => false
index => "logstash-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
I tried to :
- increase the "congestion_threshold" of lumberjack.
- increase the heap size of Elasticsearch to 2g
- increase the heap size of Logstash to 2g
- delete old index in Elasticsearch
This doesn't solve the issue.
What can I do ?