BottleNeck in local beats input

Elastic Stack Logstash
1

Hello,

I'm currently facing a bottleneck with Logstash. Currently my project is a single linux box running the FELK stack. I have determined that Logstash is a bottleneck by having filebeats write to a file (where I get 100mb per second). But when I connect to my Logstash beats port the rate drops down to nearly a fourth.

What can I do to improve performance here?

2

How do you know Logstash and not Elasticsearch is the bottleneck? Have you configured Logstash to write to file as well?

3

I can't be certain it's not an elasticsearch issue. That's a good point. Logstash doesn't write to a file. It's simply writes to ES

4

If you change it to write to file and throughput increases you have an indication that it is Elasticsearch you need to look at and not Logstash. If Elasticsearch is the limiting factor there is no point tuning Logstash.

5

Right. I will get back to you on this approach. My team has been testing a different architecture which has led to a different problem. Hoping to come back to this shortly.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.