Multiple Logstash instances vs Filebeats - Architecture


(Doug Mc Donald) #1

I'm trying to establish the best architecture for our elastic stack implementation.

We have two distinct networks (lets call them internal and external) and several web / db / application servers (approx 10) on each of these networks.

I would like to consume IIS logs, our rabbitMQ messages and some other bits and bobs from machines in both networks and send them to a single server on the internal network where my elastic and kibana installation are located.

For the servers on both the internal and external networks I can see two main ways to get the logs sent to elastic.

#1 - Setup logstash on each server and send the output to the elastic server on the internal network.

#2 - Setup filebeats on each server and send the logs to a single server running logstash (this could be the same box that hosts elastic and kibana)

I'm unsure of the pros and cons of these approaches at the moment. I believe the correct approach is to use Filebeats, but I'm unaware why I wouldn't just put logstash in multiple places as it seems like I would be better distributing the processing of logs.
Then again, perhaps having one logstash with 20-30 inputs isn't a problem?

Interested in any thoughts or guidance in this area.


(Magnus B├Ąck) #2

I prefer running low-overhead Filebeat instances on edge servers so that they can just offload logs elsewhere as quickly and non-intrusively as possible.

Then again, perhaps having one logstash with 20-30 inputs isn't a problem?

That's indeed not a problem, but why would you need to have 20-30 inputs?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.