Autoscale logstash in AWS

Elastic Stack Logstash
1

i am using logstash-input-plugin with persistent queues and fowarding my logs to Elasticsearch and a small subset of logs are sent to s3.

Is there any best practice for auto scaling my logstash instances. I am deploying them in aws.
Can i autoscale logstash based on number of events in persistent queues or some other better way to figure out when i need to add/remove instances. maybe based on response codes or maybe based on requests counts

1 Like
2

I don't have a list of "best practices", but I'm glad to share a couple "gotchas":

The persistent queue doesn't mesh well with ephemeral nodes; when Logstash gets a shutdown signal, an attempt to drain the queues will be made (unless configured otherwise), but when the node gets decommissioned, it can be tricky to ensure that the node doesn't shut down until all locally-queued events are processed.

Additionally, many of the Logstash inputs are designed as "listener"s, meaning some form of intelligent load-balancer may be needed; load-balancers are great at short-lived connections, but can be problematic for long-lived connections, so the best practice really depends on your specific use-case.

Logstash does expose monitoring APIs on the local loopback interface (e.g., localhost; not exposed to other network interfaces for security reasons), which you may be able to hook into to get state.

1 Like
3

Are there known implementations for Autoscaling on logstash.
i tried to search and most of them are auto scaling when logstash is configured in shipper/indexer arch.

i am trying to see if other people have tried autoscaling logstash with just indexer based on its own metrics.
i am pretty sure it is tricky and thats the reason why i am thinking of best practices there. I hope others agree autoscaling is required to reduce costs and also ensure optimal performance.

1 Like
4

any reference to clog posts or some guidance will be appreciated.. :slight_smile:

5

I'm interested in this as well.

I've been trying to determine the right metric to monitor to determine if another node should be spun up behind a load balancer.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.