Hi,
ubuntu 16.04 lts
logstash 2.4.0
elasticsearch 2.4.1, Build: c67dc32/2016-09-27T18:57:55Z
JVM: 1.8.0_91
let's assume, i have 2 systems (A and B). System A is running somewhere and has an ELK stack on top with an index created out of log files. I do not have access to the logs of system A but i'm able to access to ELK stack there and want to "replicate" the index and use it on system B (also ELK stack) for further investigations, enriching of events and stuff.
When i run logstash via shell like this:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/my_config.conf
i always get an out of memory error (also tried with max. heap of 4G):
Settings: Default pipeline workers: 8 Pipeline main started java.lang.OutOfMemoryError: Java heap space Dumping heap to /opt/logstash/heapdump.hprof ... Unable to create /opt/logstash/heapdump.hprof: File exists Pipeline main has been shutdown Error: Your application used more memory than the safety cap of 1G. Specify -J-Xmx####m to increase it (#### = cap size in MB). Specify -w for full OutOfMemoryError stack trace
My understanding is, that logstash will run once to retrieve the input and then shut down.
So when i want to retrieve the input in a streaming way, i have to run logstash periodically and use a special query to limit the input?
My logstash config looks like the following:
input {
elasticsearch {
hosts => ["somehostlocatedsomewhere:9200"]
index => "business"
}
}
filter {
}
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "business-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
Is there any way to prevent my out of memory problem?
I guess, doing it this way logstash tries to retrieve the whole index from system A into its memory and writes it afterwards to disk?
I tried it with a smaller index and this worked with no problem at all.
I appreciate any feedback on my issue also suggestions how to solve my approach.
Regards