Version 2,4 in case it matters.
The documentation really only mentions scaling horizontally. I'm in a position where I can get access to very "large" machines with 12-cores and 64GB+ of RAM. I'm just wondering if there are any general guidelines for scaling vertically, If there are any architectural limitations to Logstash that will make having very "big" instances of Logstash not as ideal as having many "smaller" instances.
Ultimately the best thing to do will be to do your own benchmarking. However, all of my testing leads me to the conclusion that Logstash scales poorly vertically. Given a machine with 12-cores I would expect you to have greater overall throughput running three instances of Logstash allocating 4 cores for each.
Often when sizing an Elastic Stack deployment there is a lot of focus put on the number of Elasticsearch nodes required. However the collection/ingest layer requires far more attention and planning to scale well.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.