Logstash Memory Issue


(Jimmy Kang) #1

I am using mutate to convert my json tags fields from an array to string so that it can be compared in output.

However when I put this in, I am getting some errors involving memory:

WARN: kafka.consumer.ConsumerFetcherThread: [ConsumerFetcherThread-Consumer10_9801a78e4f7f-1483048655512-36c5a8dd-0-2], Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 0; ClientId: Consumer10; ReplicaId: -1; MaxWait: 100 ms; MinBytes: 1 bytes; RequestInfo: [eilogs,46] -> PartitionFetchInfo(36918488,1048576),[eilogs,11] -> PartitionFetchInfo(50059788,1048576),[eilogs,24] -> PartitionFetchInfo(34884974,1048576),[eilogs,4] -> PartitionFetchInfo(37852143,1048576),[eilogs,6] -> PartitionFetchInfo(21304662,1048576),[eilogs,49] -> PartitionFetchInfo(17929045,1048576),[eilogs,29] -> PartitionFetchInfo(21866035,1048576),[eilogs,34] -> PartitionFetchInfo(26594259,1048576),[eilogs,41] -> PartitionFetchInfo(25539852,1048576),[eilogs,21] -> PartitionFetchInfo(25720844,1048576),[eilogs,44] -> PartitionFetchInfo(23445520,1048576),[eilogs,19] -> PartitionFetchInfo(23317766,1048576),[eilogs,26] -> PartitionFetchInfo(19740822,1048576),[eilogs,39] -> PartitionFetchInfo(25690072,1048576),[eilogs,16] -> PartitionFetchInfo(18585849,1048576),[eilogs,1] -> PartitionFetchInfo(30710821,1048576),[eilogs,36] -> PartitionFetchInfo(25314804,1048576),[eilogs,31] -> PartitionFetchInfo(31193287,1048576). Possible cause: java.lang.OutOfMemoryError: Java heap space

Below is my conf file:

filter {
    mutate {
    	convert => {"tags" => "string"}
    }
}   
output {

	mongodb {

	uri =>  "url"
	collection => "test"
   	 	database => "test"}

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.