I have been using a single vm with the whole elk stack on one machine, 6 cores and 20Gb of ram and all is well.
I am looking to scale up the amount of data we store however the actual rate of messages will be static, just looking to keep months of data instead of days.
On this basis I believe the bottleneck is not going to be with logstash (Messages per second is not going up) but with elasticsearch.
I can scale vertically and just give the instance more CPU, disk IO/space and memory, (simple changes to do) or I could look at scaling out into multiple VMs - would give some redundancy but also wastes more disk space - probably dont want to do this unless I have to as we have no need for HA, messages can que during reboots or problems. Does anyone have any real world examples of large elastic instances? How big is too big?