When I started parsing my logs with ELK stack, the speed was good but sometimes, I was able to parse 50000 logs in a minute and sometimes, it was as low as 4000 in a minute. I parsed about 2.5 million data in two and a half hours. After parsing this data, the ELK stack has slowed down.
I am now getting a speed of 1000 logs per 10 minutes. I don't understand what's wrong with it. My elasticsearch server has 64GB RAM with 31GB as memory heap. So it shouldn't give such bad performance! Please help me out here. Thanks in advance!