Hi guys, thanks for noticing this question. I posted a question about logstash input speed a couple of days ago, and it turns out the problem is because of I'm using aws. I realized that if I use Amazon's service, the logstash can be amazingly slow.
When testing the performance, I used a generator to generate logs as input, and write these logs to a local file as output. The throughput on my macbook pro is around 71,000/sec, with a dual core i5 processor and SSD. When running the same .conf file with the same default logstash configuration on a AWS m3.xlarge node, which has 4 core and 16GM Ram with SSD, I only got around 1,2000/sec throughput.
The m3.xlarge should be more powerful than my macbook pro, but it is 6 times slower. I have no idea why it's so slow. I would be appreciated if anyone can help me out. Thanks a lot.