I have copied the output statement below. It turns out to be related to using an IF statement in the output section of logstash. I wanted to have separate indexes based on the source type (set as a the prefix "myapp"), however when this put in place, I get what appears to be a memory leak. When I remove this and go with a static index name, logstash does not run out of memory.
Is there a better way to separate indexes based on type?
if [type] =~ /^myapp-/ {
elasticsearch {
hosts => ["localhost"]
sniffing => true
manage_template => false
index => "myapp-filebeat-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
else {
elasticsearch {
hosts => ["localhost"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
stdout { codec => rubydebug }
}