Scaling ELK stack

Elastic Stack Logstash
1

Hi,

I have ELK stack on AWS. It has capacity of around 5000 UDP msg/sec.

I want to scale it up to capacity of 20,000 msg/sec. What would be the recommended architecture for ELK stack.

Will it help If I keep N number of small instance installed logstash on all instance behind load balancer and keep Elasticsarch cluster on all instances or I keep bigger single instance.

And what option is better for future upgrade. ?

2

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats feels left out :wink:

If you are running Elasticsearch and Logstash together, splitting them out is definitely the first step. Using a load balancer also helps.

3

Thank you :),
Will definitely keep Logstash and Elasticsearch on separate machines .

What do you suggest about architecture. I currently run on 16 GB of memory, 4 core with capacity of around 4500 to 5000 msg/sec.

Do we just upgrade machine or we go for multiple smaller instances ?

4

You shouldn't need that much memory for Logstash, but it is CPU heavy. Having a few smaller ones is likely going to be more effective.

5

Thank you for help :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.