Logstash Architecture at Scale

[warkolm (Mark Walkom)]

@misterjulien

Should each instance be in its own server? (individual shippers, queue, and indexer) Or should the message queue and indexer be on the same server? Can/Should you have multiple Shippers on the same server?

Up to you and your resources really. If you aren't finding a single instance uses all the resources on the box then start another!

@misterjulien

Also, what's a good way to monitor the performance of the Logstash cluster? (events per second, dropped messages in the queue, memory use, etc)

You can use the metrics filter, but at the moment we don't have any monitoring visibility into LS. It's coming very soon though!